Improve caching. (#431)

* Move the caching module to core.

Add an in memory cache that for caching function and method results
during an energy management run (optimization run). Two decorators
are provided for methods and functions.

* Improve the file cache store by load and save functions.

Make EOS load the cache file store on startup and save it on shutdown.
Add a cyclic task that cleans the cache file store from outdated cache files.

* Improve startup of EOSdash by EOS

Make EOS starting EOSdash adhere to path configuration given in EOS.
The whole environment from EOS is now passed to EOSdash.
Should also prevent test errors due to unwanted/ wrong config file creation.

Both servers now provide a health endpoint that can be used to detect whether
the server is running. This is also used for testing now.

* Improve startup of EOS

EOS now has got an energy management task that runs shortly after startup.
It tries to execute energy management runs with predictions newly fetched
or initialized from cached data on first run.

* Improve shutdown of EOS

EOS has now a shutdown task that shuts EOS down gracefully with some
time delay to allow REST API requests for shutdwon or restart to be fully serviced.

* Improve EMS

Add energy management task for repeated energy management controlled by
startup delay and interval configuration parameters.
Translate EnergieManagementSystem to english EnergyManagement.

* Add administration endpoints

  - endpoints to control caching from REST API.
  - endpoints to control server restart (will not work on Windows) and shutdown from REST API

* Improve doc generation

Use "\n" linenend convention also on Windows when generating doc files.
Replace Windows specific 127.0.0.1 address by standard 0.0.0.0.

* Improve test support (to be able to test caching)

  - Add system test option to pytest for running tests with "real" resources
  - Add new test fixture to start server for test class and test function
  - Make kill signal adapt to Windows/ Linux
  - Use consistently "\n" for lineends when writing text files in  doc test
  - Fix test_logging under Windows
  - Fix conftest config_default_dirs test fixture under Windows

From @Lasall

* Improve Windows support

 - Use 127.0.0.1 as default config host (model defaults) and
   addionally redirect 0.0.0.0 to localhost on Windows (because default
   config file still has 0.0.0.0).
 - Update install/startup instructions as package installation is
   required atm.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

CONTRIBUTING.md

@@ -33,6 +33,7 @@ See also [README.md](README.md).
 python -m venv .venv
 source .venv/bin/activate
 pip install -r requirements-dev.txt
+pip install -e .
 ```
 
 Install make to get access to helpful shortcuts (documentation generation, manual formatting, etc.).

README.md

@@ -23,13 +23,15 @@ Linux:
 ```bash
 python -m venv .venv
 .venv/bin/pip install -r requirements.txt
+.venv/bin/pip install -e .
 ```
 
 Windows:
 
 ```cmd
 python -m venv .venv
- .venv\Scripts\pip install -r requirements.txt
+.venv\Scripts\pip install -r requirements.txt
+.venv\Scripts\pip install -e .
 ```
 
 Finally, start the EOS server:

docs/_generated/config.md

@@ -27,12 +27,10 @@ Validators:
 | ---- | -------------------- | ---- | --------- | ------- | ----------- |
 | data_folder_path | `EOS_GENERAL__DATA_FOLDER_PATH` | `Optional[pathlib.Path]` | `rw` | `None` | Path to EOS data directory. |
 | data_output_subpath | `EOS_GENERAL__DATA_OUTPUT_SUBPATH` | `Optional[pathlib.Path]` | `rw` | `output` | Sub-path for the EOS output data directory. |
-| data_cache_subpath | `EOS_GENERAL__DATA_CACHE_SUBPATH` | `Optional[pathlib.Path]` | `rw` | `cache` | Sub-path for the EOS cache data directory. |
 | latitude | `EOS_GENERAL__LATITUDE` | `Optional[float]` | `rw` | `52.52` | Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°) |
 | longitude | `EOS_GENERAL__LONGITUDE` | `Optional[float]` | `rw` | `13.405` | Longitude in decimal degrees, within -180 to 180 (°) |
 | timezone | | `Optional[str]` | `ro` | `N/A` | Compute timezone based on latitude and longitude. |
 | data_output_path | | `Optional[pathlib.Path]` | `ro` | `N/A` | Compute data_output_path based on data_folder_path. |
-| data_cache_path | | `Optional[pathlib.Path]` | `ro` | `N/A` | Compute data_cache_path based on data_folder_path. |
 | config_folder_path | | `Optional[pathlib.Path]` | `ro` | `N/A` | Path to EOS configuration directory. |
 | config_file_path | | `Optional[pathlib.Path]` | `ro` | `N/A` | Path to EOS configuration file. |
 :::
@@ -46,7 +44,6 @@ Validators:
        "general": {
            "data_folder_path": null,
            "data_output_subpath": "output",
-           "data_cache_subpath": "cache",
            "latitude": 52.52,
            "longitude": 13.405
        }
@@ -62,18 +59,66 @@ Validators:
        "general": {
            "data_folder_path": null,
            "data_output_subpath": "output",
-           "data_cache_subpath": "cache",
            "latitude": 52.52,
            "longitude": 13.405,
            "timezone": "Europe/Berlin",
            "data_output_path": null,
-           "data_cache_path": null,
            "config_folder_path": "/home/user/.config/net.akkudoktoreos.net",
            "config_file_path": "/home/user/.config/net.akkudoktoreos.net/EOS.config.json"
        }
    }
 ```
 
+## Cache Configuration
+
+:::{table} cache
+:widths: 10 20 10 5 5 30
+:align: left
+
+| Name | Environment Variable | Type | Read-Only | Default | Description |
+| ---- | -------------------- | ---- | --------- | ------- | ----------- |
+| subpath | `EOS_CACHE__SUBPATH` | `Optional[pathlib.Path]` | `rw` | `cache` | Sub-path for the EOS cache data directory. |
+| cleanup_interval | `EOS_CACHE__CLEANUP_INTERVAL` | `float` | `rw` | `300` | Intervall in seconds for EOS file cache cleanup. |
+:::
+
+### Example Input/Output
+
+```{eval-rst}
+.. code-block:: json
+
+   {
+       "cache": {
+           "subpath": "cache",
+           "cleanup_interval": 300.0
+       }
+   }
+```
+
+## Energy Management Configuration
+
+:::{table} ems
+:widths: 10 20 10 5 5 30
+:align: left
+
+| Name | Environment Variable | Type | Read-Only | Default | Description |
+| ---- | -------------------- | ---- | --------- | ------- | ----------- |
+| startup_delay | `EOS_EMS__STARTUP_DELAY` | `float` | `rw` | `5` | Startup delay in seconds for EOS energy management runs. |
+| interval | `EOS_EMS__INTERVAL` | `Optional[float]` | `rw` | `None` | Intervall in seconds between EOS energy management runs. |
+:::
+
+### Example Input/Output
+
+```{eval-rst}
+.. code-block:: json
+
+   {
+       "ems": {
+           "startup_delay": 5.0,
+           "interval": 300.0
+       }
+   }
+```
+
 ## Logging Configuration
 
 :::{table} logging
@@ -889,10 +934,17 @@ Attributes:
        "general": {
            "data_folder_path": null,
            "data_output_subpath": "output",
-           "data_cache_subpath": "cache",
            "latitude": 52.52,
            "longitude": 13.405
        },
+       "cache": {
+           "subpath": "cache",
+           "cleanup_interval": 300.0
+       },
+       "ems": {
+           "startup_delay": 5.0,
+           "interval": 300.0
+       },
        "logging": {
            "level": "INFO"
        },

docs/_generated/openapi.md

@@ -166,6 +166,127 @@ Note:
 
 ---
 
+## GET /v1/admin/cache
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_cache_get_v1_admin_cache_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_cache_get_v1_admin_cache_get)
+
+Fastapi Admin Cache Get
+
+```
+Current cache management data.
+
+Returns:
+    data (dict): The management data.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
+## POST /v1/admin/cache/clear
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_cache_clear_post_v1_admin_cache_clear_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_cache_clear_post_v1_admin_cache_clear_post)
+
+Fastapi Admin Cache Clear Post
+
+```
+Clear the cache from expired data.
+
+Deletes expired cache files.
+
+Args:
+    clear_all (Optional[bool]): Delete all cached files. Default is False.
+
+Returns:
+    data (dict): The management data after cleanup.
+```
+
+**Parameters**:
+
+- `clear_all` (query, optional): No description provided.
+
+**Responses**:
+
+- **200**: Successful Response
+
+- **422**: Validation Error
+
+---
+
+## POST /v1/admin/cache/load
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_cache_load_post_v1_admin_cache_load_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_cache_load_post_v1_admin_cache_load_post)
+
+Fastapi Admin Cache Load Post
+
+```
+Load cache management data.
+
+Returns:
+    data (dict): The management data that was loaded.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
+## POST /v1/admin/cache/save
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_cache_save_post_v1_admin_cache_save_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_cache_save_post_v1_admin_cache_save_post)
+
+Fastapi Admin Cache Save Post
+
+```
+Save the current cache management data.
+
+Returns:
+    data (dict): The management data that was saved.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
+## POST /v1/admin/server/restart
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_server_restart_post_v1_admin_server_restart_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_server_restart_post_v1_admin_server_restart_post)
+
+Fastapi Admin Server Restart Post
+
+```
+Restart the server.
+
+Restart EOS properly by starting a new instance before exiting the old one.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
+## POST /v1/admin/server/shutdown
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_admin_server_shutdown_post_v1_admin_server_shutdown_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_admin_server_shutdown_post_v1_admin_server_shutdown_post)
+
+Fastapi Admin Server Shutdown Post
+
+```
+Shutdown the server.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
 ## GET /v1/config
 
 **Links**: [local](http://localhost:8503/docs#/default/fastapi_config_get_v1_config_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_get_v1_config_get)
@@ -238,14 +359,14 @@ Returns:
 
 ---
 
-## PUT /v1/config/reset
+## POST /v1/config/reset
 
-**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_update_post_v1_config_reset_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_update_post_v1_config_reset_put)
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_reset_post_v1_config_reset_post), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_reset_post_v1_config_reset_post)
 
-Fastapi Config Update Post
+Fastapi Config Reset Post
 
 ```
-Reset the configuration to the EOS configuration file.
+Reset the configuration.
 
 Returns:
     configuration (ConfigEOS): The current configuration after update.
@@ -321,6 +442,22 @@ Returns:
 
 ---
 
+## GET /v1/health
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_health_get_v1_health_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_health_get_v1_health_get)
+
+Fastapi Health Get
+
+```
+Health check endpoint to verify that the EOS server is alive.
+```
+
+**Responses**:
+
+- **200**: Successful Response
+
+---
+
 ## PUT /v1/measurement/data
 
 **Links**: [local](http://localhost:8503/docs#/default/fastapi_measurement_data_put_v1_measurement_data_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_measurement_data_put_v1_measurement_data_put)
@@ -537,6 +674,56 @@ Merge the measurement of given key and value into EOS measurements at given date
 
 ---
 
+## PUT /v1/prediction/import/{provider_id}
+
+**Links**: [local](http://localhost:8503/docs#/default/fastapi_prediction_import_provider_v1_prediction_import__provider_id__put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_prediction_import_provider_v1_prediction_import__provider_id__put)
+
+Fastapi Prediction Import Provider
+
+```
+Import prediction for given provider ID.
+
+Args:
+    provider_id: ID of provider to update.
+    data: Prediction data.
+    force_enable: Update data even if provider is disabled.
+        Defaults to False.
+```
+
+**Parameters**:
+
+- `provider_id` (path, required): Provider ID.
+
+- `force_enable` (query, optional): No description provided.
+
+**Request Body**:
+
+- `application/json`: {
+  "anyOf": [
+    {
+      "$ref": "#/components/schemas/PydanticDateTimeDataFrame"
+    },
+    {
+      "$ref": "#/components/schemas/PydanticDateTimeData"
+    },
+    {
+      "type": "object"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "title": "Data"
+}
+
+**Responses**:
+
+- **200**: Successful Response
+
+- **422**: Validation Error
+
+---
+
 ## GET /v1/prediction/keys
 
 **Links**: [local](http://localhost:8503/docs#/default/fastapi_prediction_keys_get_v1_prediction_keys_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_prediction_keys_get_v1_prediction_keys_get)
@@ -580,7 +767,7 @@ Args:
 
 - `end_datetime` (query, optional): Ending datetime (exclusive).
 
-- `interval` (query, optional): Time duration for each interval.
+- `interval` (query, optional): Time duration for each interval. Defaults to 1 hour.
 
 **Responses**:
 

docs/akkudoktoreos/prediction.md

@@ -141,9 +141,12 @@ The prediction key for the electricity price forecast data is:
 - `elecprice_marketprice_wh`: Electricity market price per Wh (€/Wh).
 
 The electricity proce forecast data must be provided in one of the formats described in
-<project:#prediction-import-providers>. The data source must be given in the
+<project:#prediction-import-providers>. The data source can be given in the
 `import_file_path` or `import_json` configuration option.
 
+The data may additionally or solely be provided by the
+**PUT** `/v1/prediction/import/ElecPriceImport` endpoint.
+
 ## Load Prediction
 
 Prediction keys:
@@ -184,9 +187,12 @@ The prediction keys for the load forecast data are:
 - `load_mean_adjusted`: Predicted load mean value adjusted by load measurement (W).
 
 The load forecast data must be provided in one of the formats described in
-<project:#prediction-import-providers>. The data source must be given in the `loadimport_file_path`
+<project:#prediction-import-providers>. The data source can be given in the `loadimport_file_path`
 or `loadimport_json` configuration option.
 
+The data may additionally or solely be provided by the
+**PUT** `/v1/prediction/import/LoadImport` endpoint.
+
 ## PV Power Prediction
 
 Prediction keys:
@@ -362,9 +368,12 @@ The prediction keys for the PV forecast data are:
 - `pvforecast_dc_power`: Total AC power (W).
 
 The PV forecast data must be provided in one of the formats described in
-<project:#prediction-import-providers>. The data source must be given in the
+<project:#prediction-import-providers>. The data source can be given in the
 `import_file_path` or `import_json` configuration option.
 
+The data may additionally or solely be provided by the
+**PUT** `/v1/prediction/import/PVForecastImport` endpoint.
+
 ## Weather Prediction
 
 Prediction keys:
@@ -460,7 +469,7 @@ The `WeatherImport` provider is designed to import weather forecast data from a
 string. An external entity should update the file or JSON string whenever new prediction data
 becomes available.
 
-The prediction keys for the PV forecast data are:
+The prediction keys for the weather forecast data are:
 
 - `weather_dew_point`: Dew Point (°C)
 - `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
@@ -486,5 +495,8 @@ The prediction keys for the PV forecast data are:
 - `weather_wind_speed`: Wind Speed (kmph)
 
 The PV forecast data must be provided in one of the formats described in
-<project:#prediction-import-providers>. The data source must be given in the
+<project:#prediction-import-providers>. The data source can be given in the
 `import_file_path` or `import_json` configuration option.
+
+The data may additionally or solely be provided by the
+**PUT** `/v1/prediction/import/WeatherImport` endpoint.

docs/develop/getting_started.md

@@ -19,6 +19,7 @@ Install the dependencies in a virtual environment:
 
         python -m venv .venv
         .venv\Scripts\pip install -r requirements.txt
+        .venv\Scripts\pip install -e .
 
   .. tab:: Linux
 
@@ -26,6 +27,7 @@ Install the dependencies in a virtual environment:
 
         python -m venv .venv
         .venv/bin/pip install -r requirements.txt
+        .venv/bin/pip install -e .
 
 ```
 

openapi.json

@@ -106,10 +106,44 @@
                 "title": "BaseBatteryParameters",
                 "type": "object"
             },
+            "CacheCommonSettings": {
+                "description": "Cache Configuration.",
+                "properties": {
+                    "cleanup_interval": {
+                        "default": 300,
+                        "description": "Intervall in seconds for EOS file cache cleanup.",
+                        "title": "Cleanup Interval",
+                        "type": "number"
+                    },
+                    "subpath": {
+                        "anyOf": [
+                            {
+                                "format": "path",
+                                "type": "string"
+                            },
+                            {
+                                "type": "null"
+                            }
+                        ],
+                        "default": "cache",
+                        "description": "Sub-path for the EOS cache data directory.",
+                        "title": "Subpath"
+                    }
+                },
+                "title": "CacheCommonSettings",
+                "type": "object"
+            },
             "ConfigEOS": {
                 "additionalProperties": false,
                 "description": "Singleton configuration handler for the EOS application.\n\nConfigEOS extends `SettingsEOS` with support for  default configuration paths and automatic\ninitialization.\n\n`ConfigEOS` ensures that only one instance of the class is created throughout the application,\nallowing consistent access to EOS configuration settings. This singleton instance loads\nconfiguration data from a predefined set of directories or creates a default configuration if\nnone is found.\n\nInitialization Process:\n  - Upon instantiation, the singleton instance attempts to load a configuration file in this order:\n    1. The directory specified by the `EOS_CONFIG_DIR` environment variable\n    2. The directory specified by the `EOS_DIR` environment variable.\n    3. A platform specific default directory for EOS.\n    4. The current working directory.\n  - The first available configuration file found in these directories is loaded.\n  - If no configuration file is found, a default configuration file is created in the platform\n    specific default directory, and default settings are loaded into it.\n\nAttributes from the loaded configuration are accessible directly as instance attributes of\n`ConfigEOS`, providing a centralized, shared configuration object for EOS.\n\nSingleton Behavior:\n  - This class uses the `SingletonMixin` to ensure that all requests for `ConfigEOS` return\n    the same instance, which contains the most up-to-date configuration. Modifying the configuration\n    in one part of the application reflects across all references to this class.\n\nAttributes:\n    config_folder_path (Optional[Path]): Path to the configuration directory.\n    config_file_path (Optional[Path]): Path to the configuration file.\n\nRaises:\n    FileNotFoundError: If no configuration file is found, and creating a default configuration fails.\n\nExample:\n    To initialize and access configuration attributes (only one instance is created):\n    ```python\n    config_eos = ConfigEOS()  # Always returns the same instance\n    print(config_eos.prediction.hours)  # Access a setting from the loaded configuration\n    ```",
                 "properties": {
+                    "cache": {
+                        "$ref": "#/components/schemas/CacheCommonSettings",
+                        "default": {
+                            "cleanup_interval": 300.0,
+                            "subpath": "cache"
+                        }
+                    },
                     "devices": {
                         "$ref": "#/components/schemas/DevicesCommonSettings",
                         "default": {}
@@ -118,12 +152,17 @@
                         "$ref": "#/components/schemas/ElecPriceCommonSettings",
                         "default": {}
                     },
+                    "ems": {
+                        "$ref": "#/components/schemas/EnergyManagementCommonSettings",
+                        "default": {
+                            "startup_delay": 5.0
+                        }
+                    },
                     "general": {
                         "$ref": "#/components/schemas/GeneralSettings-Output",
                         "default": {
                             "config_file_path": "/home/user/.config/net.akkudoktoreos.net/EOS.config.json",
                             "config_folder_path": "/home/user/.config/net.akkudoktoreos.net",
-                            "data_cache_subpath": "cache",
                             "data_output_subpath": "output",
                             "latitude": 52.52,
                             "longitude": 13.405,
@@ -548,7 +587,36 @@
                 "title": "ElectricVehicleResult",
                 "type": "object"
             },
-            "EnergieManagementSystemParameters": {
+            "EnergyManagementCommonSettings": {
+                "description": "Energy Management Configuration.",
+                "properties": {
+                    "interval": {
+                        "anyOf": [
+                            {
+                                "type": "number"
+                            },
+                            {
+                                "type": "null"
+                            }
+                        ],
+                        "description": "Intervall in seconds between EOS energy management runs.",
+                        "examples": [
+                            "300"
+                        ],
+                        "title": "Interval"
+                    },
+                    "startup_delay": {
+                        "default": 5,
+                        "description": "Startup delay in seconds for EOS energy management runs.",
+                        "minimum": 1.0,
+                        "title": "Startup Delay",
+                        "type": "number"
+                    }
+                },
+                "title": "EnergyManagementCommonSettings",
+                "type": "object"
+            },
+            "EnergyManagementParameters": {
                 "additionalProperties": false,
                 "properties": {
                     "einspeiseverguetung_euro_pro_wh": {
@@ -603,7 +671,7 @@
                     "preis_euro_pro_wh_akku",
                     "gesamtlast"
                 ],
-                "title": "EnergieManagementSystemParameters",
+                "title": "EnergyManagementParameters",
                 "type": "object"
             },
             "ForecastResponse": {
@@ -640,20 +708,6 @@
             "GeneralSettings-Input": {
                 "description": "Settings for common configuration.\n\nGeneral configuration to set directories of cache and output files and system location (latitude\nand longitude).\nValidators ensure each parameter is within a specified range. A computed property, `timezone`,\ndetermines the time zone based on latitude and longitude.\n\nAttributes:\n    latitude (Optional[float]): Latitude in degrees, must be between -90 and 90.\n    longitude (Optional[float]): Longitude in degrees, must be between -180 and 180.\n\nProperties:\n    timezone (Optional[str]): Computed time zone string based on the specified latitude\n        and longitude.\n\nValidators:\n    validate_latitude (float): Ensures `latitude` is within the range -90 to 90.\n    validate_longitude (float): Ensures `longitude` is within the range -180 to 180.",
                 "properties": {
-                    "data_cache_subpath": {
-                        "anyOf": [
-                            {
-                                "format": "path",
-                                "type": "string"
-                            },
-                            {
-                                "type": "null"
-                            }
-                        ],
-                        "default": "cache",
-                        "description": "Sub-path for the EOS cache data directory.",
-                        "title": "Data Cache Subpath"
-                    },
                     "data_folder_path": {
                         "anyOf": [
                             {
@@ -750,34 +804,6 @@
                         "readOnly": true,
                         "title": "Config Folder Path"
                     },
-                    "data_cache_path": {
-                        "anyOf": [
-                            {
-                                "format": "path",
-                                "type": "string"
-                            },
-                            {
-                                "type": "null"
-                            }
-                        ],
-                        "description": "Compute data_cache_path based on data_folder_path.",
-                        "readOnly": true,
-                        "title": "Data Cache Path"
-                    },
-                    "data_cache_subpath": {
-                        "anyOf": [
-                            {
-                                "format": "path",
-                                "type": "string"
-                            },
-                            {
-                                "type": "null"
-                            }
-                        ],
-                        "default": "cache",
-                        "description": "Sub-path for the EOS cache data directory.",
-                        "title": "Data Cache Subpath"
-                    },
                     "data_folder_path": {
                         "anyOf": [
                             {
@@ -870,7 +896,6 @@
                 "required": [
                     "timezone",
                     "data_output_path",
-                    "data_cache_path",
                     "config_folder_path",
                     "config_file_path"
                 ],
@@ -1361,7 +1386,7 @@
                         ]
                     },
                     "ems": {
-                        "$ref": "#/components/schemas/EnergieManagementSystemParameters"
+                        "$ref": "#/components/schemas/EnergyManagementParameters"
                     },
                     "inverter": {
                         "anyOf": [
@@ -2294,6 +2319,17 @@
                 "additionalProperties": false,
                 "description": "Settings for all EOS.\n\nUsed by updating the configuration with specific settings only.",
                 "properties": {
+                    "cache": {
+                        "anyOf": [
+                            {
+                                "$ref": "#/components/schemas/CacheCommonSettings"
+                            },
+                            {
+                                "type": "null"
+                            }
+                        ],
+                        "description": "Cache Settings"
+                    },
                     "devices": {
                         "anyOf": [
                             {
@@ -2302,7 +2338,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Devices Settings"
                     },
                     "elecprice": {
                         "anyOf": [
@@ -2312,7 +2349,19 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Electricity Price Settings"
+                    },
+                    "ems": {
+                        "anyOf": [
+                            {
+                                "$ref": "#/components/schemas/EnergyManagementCommonSettings"
+                            },
+                            {
+                                "type": "null"
+                            }
+                        ],
+                        "description": "Energy Management Settings"
                     },
                     "general": {
                         "anyOf": [
@@ -2322,7 +2371,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "General Settings"
                     },
                     "load": {
                         "anyOf": [
@@ -2332,7 +2382,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Load Settings"
                     },
                     "logging": {
                         "anyOf": [
@@ -2342,7 +2393,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Logging Settings"
                     },
                     "measurement": {
                         "anyOf": [
@@ -2352,7 +2404,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Measurement Settings"
                     },
                     "optimization": {
                         "anyOf": [
@@ -2362,7 +2415,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Optimization Settings"
                     },
                     "prediction": {
                         "anyOf": [
@@ -2372,7 +2426,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Prediction Settings"
                     },
                     "pvforecast": {
                         "anyOf": [
@@ -2382,7 +2437,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "PV Forecast Settings"
                     },
                     "server": {
                         "anyOf": [
@@ -2392,7 +2448,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Server Settings"
                     },
                     "utils": {
                         "anyOf": [
@@ -2402,7 +2459,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Utilities Settings"
                     },
                     "weather": {
                         "anyOf": [
@@ -2412,7 +2470,8 @@
                             {
                                 "type": "null"
                             }
-                        ]
+                        ],
+                        "description": "Weather Settings"
                     }
                 },
                 "title": "SettingsEOS",
@@ -3060,6 +3119,172 @@
                 ]
             }
         },
+        "/v1/admin/cache": {
+            "get": {
+                "description": "Current cache management data.\n\nReturns:\n    data (dict): The management data.",
+                "operationId": "fastapi_admin_cache_get_v1_admin_cache_get",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Cache Get V1 Admin Cache Get",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Admin Cache Get",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
+        "/v1/admin/cache/clear": {
+            "post": {
+                "description": "Clear the cache from expired data.\n\nDeletes expired cache files.\n\nArgs:\n    clear_all (Optional[bool]): Delete all cached files. Default is False.\n\nReturns:\n    data (dict): The management data after cleanup.",
+                "operationId": "fastapi_admin_cache_clear_post_v1_admin_cache_clear_post",
+                "parameters": [
+                    {
+                        "in": "query",
+                        "name": "clear_all",
+                        "required": false,
+                        "schema": {
+                            "anyOf": [
+                                {
+                                    "type": "boolean"
+                                },
+                                {
+                                    "type": "null"
+                                }
+                            ],
+                            "title": "Clear All"
+                        }
+                    }
+                ],
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Cache Clear Post V1 Admin Cache Clear Post",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    },
+                    "422": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "$ref": "#/components/schemas/HTTPValidationError"
+                                }
+                            }
+                        },
+                        "description": "Validation Error"
+                    }
+                },
+                "summary": "Fastapi Admin Cache Clear Post",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
+        "/v1/admin/cache/load": {
+            "post": {
+                "description": "Load cache management data.\n\nReturns:\n    data (dict): The management data that was loaded.",
+                "operationId": "fastapi_admin_cache_load_post_v1_admin_cache_load_post",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Cache Load Post V1 Admin Cache Load Post",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Admin Cache Load Post",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
+        "/v1/admin/cache/save": {
+            "post": {
+                "description": "Save the current cache management data.\n\nReturns:\n    data (dict): The management data that was saved.",
+                "operationId": "fastapi_admin_cache_save_post_v1_admin_cache_save_post",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Cache Save Post V1 Admin Cache Save Post",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Admin Cache Save Post",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
+        "/v1/admin/server/restart": {
+            "post": {
+                "description": "Restart the server.\n\nRestart EOS properly by starting a new instance before exiting the old one.",
+                "operationId": "fastapi_admin_server_restart_post_v1_admin_server_restart_post",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Server Restart Post V1 Admin Server Restart Post",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Admin Server Restart Post",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
+        "/v1/admin/server/shutdown": {
+            "post": {
+                "description": "Shutdown the server.",
+                "operationId": "fastapi_admin_server_shutdown_post_v1_admin_server_shutdown_post",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "title": "Response Fastapi Admin Server Shutdown Post V1 Admin Server Shutdown Post",
+                                    "type": "object"
+                                }
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Admin Server Shutdown Post",
+                "tags": [
+                    "admin"
+                ]
+            }
+        },
         "/v1/config": {
             "get": {
                 "description": "Get the current configuration.\n\nReturns:\n    configuration (ConfigEOS): The current configuration.",
@@ -3145,9 +3370,9 @@
             }
         },
         "/v1/config/reset": {
-            "put": {
-                "description": "Reset the configuration to the EOS configuration file.\n\nReturns:\n    configuration (ConfigEOS): The current configuration after update.",
-                "operationId": "fastapi_config_update_post_v1_config_reset_put",
+            "post": {
+                "description": "Reset the configuration.\n\nReturns:\n    configuration (ConfigEOS): The current configuration after update.",
+                "operationId": "fastapi_config_reset_post_v1_config_reset_post",
                 "responses": {
                     "200": {
                         "content": {
@@ -3160,7 +3385,7 @@
                         "description": "Successful Response"
                     }
                 },
-                "summary": "Fastapi Config Update Post",
+                "summary": "Fastapi Config Reset Post",
                 "tags": [
                     "config"
                 ]
@@ -3263,6 +3488,23 @@
                 ]
             }
         },
+        "/v1/health": {
+            "get": {
+                "description": "Health check endpoint to verify that the EOS server is alive.",
+                "operationId": "fastapi_health_get_v1_health_get",
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {}
+                            }
+                        },
+                        "description": "Successful Response"
+                    }
+                },
+                "summary": "Fastapi Health Get"
+            }
+        },
         "/v1/measurement/data": {
             "put": {
                 "description": "Merge the measurement data given as datetime data into EOS measurements.",
@@ -3709,6 +3951,88 @@
                 ]
             }
         },
+        "/v1/prediction/import/{provider_id}": {
+            "put": {
+                "description": "Import prediction for given provider ID.\n\nArgs:\n    provider_id: ID of provider to update.\n    data: Prediction data.\n    force_enable: Update data even if provider is disabled.\n        Defaults to False.",
+                "operationId": "fastapi_prediction_import_provider_v1_prediction_import__provider_id__put",
+                "parameters": [
+                    {
+                        "description": "Provider ID.",
+                        "in": "path",
+                        "name": "provider_id",
+                        "required": true,
+                        "schema": {
+                            "description": "Provider ID.",
+                            "title": "Provider Id",
+                            "type": "string"
+                        }
+                    },
+                    {
+                        "in": "query",
+                        "name": "force_enable",
+                        "required": false,
+                        "schema": {
+                            "anyOf": [
+                                {
+                                    "type": "boolean"
+                                },
+                                {
+                                    "type": "null"
+                                }
+                            ],
+                            "title": "Force Enable"
+                        }
+                    }
+                ],
+                "requestBody": {
+                    "content": {
+                        "application/json": {
+                            "schema": {
+                                "anyOf": [
+                                    {
+                                        "$ref": "#/components/schemas/PydanticDateTimeDataFrame"
+                                    },
+                                    {
+                                        "$ref": "#/components/schemas/PydanticDateTimeData"
+                                    },
+                                    {
+                                        "type": "object"
+                                    },
+                                    {
+                                        "type": "null"
+                                    }
+                                ],
+                                "title": "Data"
+                            }
+                        }
+                    }
+                },
+                "responses": {
+                    "200": {
+                        "content": {
+                            "application/json": {
+                                "schema": {}
+                            }
+                        },
+                        "description": "Successful Response"
+                    },
+                    "422": {
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "$ref": "#/components/schemas/HTTPValidationError"
+                                }
+                            }
+                        },
+                        "description": "Validation Error"
+                    }
+                },
+                "summary": "Fastapi Prediction Import Provider",
+                "tags": [
+                    "prediction"
+                ]
+            }
+        },
         "/v1/prediction/keys": {
             "get": {
                 "description": "Get a list of available prediction keys.",
@@ -3788,7 +4112,7 @@
                         }
                     },
                     {
-                        "description": "Time duration for each interval.",
+                        "description": "Time duration for each interval. Defaults to 1 hour.",
                         "in": "query",
                         "name": "interval",
                         "required": false,
@@ -3801,7 +4125,7 @@
                                     "type": "null"
                                 }
                             ],
-                            "description": "Time duration for each interval.",
+                            "description": "Time duration for each interval. Defaults to 1 hour.",
                             "title": "Interval"
                         }
                     }
@@ -3981,9 +4305,16 @@
                         "name": "force_update",
                         "required": false,
                         "schema": {
+                            "anyOf": [
+                                {
+                                    "type": "boolean"
+                                },
+                                {
+                                    "type": "null"
+                                }
+                            ],
                             "default": false,
-                            "title": "Force Update",
-                            "type": "boolean"
+                            "title": "Force Update"
                         }
                     },
                     {
@@ -3991,9 +4322,16 @@
                         "name": "force_enable",
                         "required": false,
                         "schema": {
+                            "anyOf": [
+                                {
+                                    "type": "boolean"
+                                },
+                                {
+                                    "type": "null"
+                                }
+                            ],
                             "default": false,
-                            "title": "Force Enable",
-                            "type": "boolean"
+                            "title": "Force Enable"
                         }
                     }
                 ],

requirements.txt

@@ -1,3 +1,4 @@
+cachebox==4.4.2
 numpy==2.2.2
 numpydantic==1.6.7
 matplotlib==3.10.0
@@ -11,6 +12,7 @@ requests==2.32.3
 pandas==2.2.3
 pendulum==3.0.0
 platformdirs==4.3.6
+psutil==6.1.1
 pvlib==0.11.2
 pydantic==2.10.6
 statsmodels==0.14.4

scripts/extract_markdown.py

@@ -150,7 +150,7 @@ def main():
 
     try:
         if args.input_file:
-            with open(args.input_file, "r", encoding="utf8") as f:
+            with open(args.input_file, "r", encoding="utf-8", newline=None) as f:
                 content = f.read()
         elif args.input:
             content = args.input
@@ -164,7 +164,7 @@ def main():
         )
         if args.output_file:
             # Write to file
-            with open(args.output_file, "w", encoding="utf8") as f:
+            with open(args.output_file, "w", encoding="utf-8", newline="\n") as f:
                 f.write(extracted_content)
         else:
             # Write to std output

scripts/generate_config_md.py

@@ -3,6 +3,7 @@
 
 import argparse
 import json
+import os
 import sys
 import textwrap
 from pathlib import Path
@@ -296,9 +297,11 @@ def main():
 
     try:
         config_md = generate_config_md(config_eos)
+        if os.name == "nt":
+            config_md = config_md.replace("127.0.0.1", "0.0.0.0").replace("\\\\", "/")
         if args.output_file:
             # Write to file
-            with open(args.output_file, "w", encoding="utf8") as f:
+            with open(args.output_file, "w", encoding="utf-8", newline="\n") as f:
                 f.write(config_md)
         else:
             # Write to std output

scripts/generate_openapi.py

@@ -16,6 +16,7 @@ Example:
 
 import argparse
 import json
+import os
 import sys
 
 from fastapi.openapi.utils import get_openapi
@@ -57,9 +58,11 @@ def main():
     try:
         openapi_spec = generate_openapi()
         openapi_spec_str = json.dumps(openapi_spec, indent=2)
+        if os.name == "nt":
+            openapi_spec_str = openapi_spec_str.replace("127.0.0.1", "0.0.0.0")
         if args.output_file:
             # Write to file
-            with open(args.output_file, "w", encoding="utf8") as f:
+            with open(args.output_file, "w", encoding="utf-8", newline="\n") as f:
                 f.write(openapi_spec_str)
         else:
             # Write to std output

scripts/generate_openapi_md.py

@@ -3,6 +3,7 @@
 
 import argparse
 import json
+import os
 import sys
 
 import git
@@ -284,9 +285,11 @@ def main():
 
     try:
         openapi_md = generate_openapi_md()
+        if os.name == "nt":
+            openapi_md = openapi_md.replace("127.0.0.1", "0.0.0.0")
         if args.output_file:
             # Write to file
-            with open(args.output_file, "w", encoding="utf8") as f:
+            with open(args.output_file, "w", encoding="utf-8", newline="\n") as f:
                 f.write(openapi_md)
         else:
             # Write to std output

single_test_prediction.py

@@ -121,30 +121,40 @@ def run_prediction(provider_id: str, verbose: bool = False) -> str:
     # Initialize the oprediction
     config_eos = get_config()
     prediction_eos = get_prediction()
-    if verbose:
-        print(f"\nProvider ID: {provider_id}")
     if provider_id in ("PVForecastAkkudoktor",):
         settings = config_pvforecast()
-        settings["pvforecast"]["provider"] = provider_id
+        forecast = "pvforecast"
     elif provider_id in ("BrightSky", "ClearOutside"):
         settings = config_weather()
-        settings["weather"]["provider"] = provider_id
+        forecast = "weather"
     elif provider_id in ("ElecPriceAkkudoktor",):
         settings = config_elecprice()
-        settings["elecprice"]["provider"] = provider_id
+        forecast = "elecprice"
     elif provider_id in ("LoadAkkudoktor",):
         settings = config_elecprice()
+        forecast = "load"
         settings["load"]["loadakkudoktor_year_energy"] = 1000
-        settings["load"]["provider"] = provider_id
     else:
         raise ValueError(f"Unknown provider '{provider_id}'.")
+    settings[forecast]["provider"] = provider_id
     config_eos.merge_settings_from_dict(settings)
 
+    provider = prediction_eos.provider_by_id(provider_id)
+
     prediction_eos.update_data()
 
     # Return result of prediction
-    provider = prediction_eos.provider_by_id(provider_id)
     if verbose:
+        print(f"\nProvider ID: {provider.provider_id()}")
+        print("----------")
+        print("\nSettings\n----------")
+        print(settings)
+        print("\nProvider\n----------")
+        print(f"elecprice.provider: {config_eos.elecprice.provider}")
+        print(f"load.provider: {config_eos.load.provider}")
+        print(f"pvforecast.provider: {config_eos.pvforecast.provider}")
+        print(f"weather.provider: {config_eos.weather.provider}")
+        print(f"enabled: {provider.enabled()}")
         for key in provider.record_keys:
             print(f"\n{key}\n----------")
             print(f"Array: {provider.key_to_array(key)}")

src/akkudoktoreos/config/config.py

@@ -22,12 +22,13 @@ from pydantic_settings import (
     PydanticBaseSettingsSource,
     SettingsConfigDict,
 )
-from pydantic_settings.sources import ConfigFileSourceMixin
 
 # settings
 from akkudoktoreos.config.configabc import SettingsBaseModel
+from akkudoktoreos.core.cachesettings import CacheCommonSettings
 from akkudoktoreos.core.coreabc import SingletonMixin
 from akkudoktoreos.core.decorators import classproperty
+from akkudoktoreos.core.emsettings import EnergyManagementCommonSettings
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.logsettings import LoggingCommonSettings
 from akkudoktoreos.core.pydantic import access_nested_value, merge_models
@@ -96,10 +97,6 @@ class GeneralSettings(SettingsBaseModel):
         default="output", description="Sub-path for the EOS output data directory."
     )
 
-    data_cache_subpath: Optional[Path] = Field(
-        default="cache", description="Sub-path for the EOS cache data directory."
-    )
-
     latitude: Optional[float] = Field(
         default=52.52,
         ge=-90.0,
@@ -128,12 +125,6 @@ class GeneralSettings(SettingsBaseModel):
         """Compute data_output_path based on data_folder_path."""
         return get_absolute_path(self.data_folder_path, self.data_output_subpath)
 
-    @computed_field  # type: ignore[prop-decorator]
-    @property
-    def data_cache_path(self) -> Optional[Path]:
-        """Compute data_cache_path based on data_folder_path."""
-        return get_absolute_path(self.data_folder_path, self.data_cache_subpath)
-
     @computed_field  # type: ignore[prop-decorator]
     @property
     def config_folder_path(self) -> Optional[Path]:
@@ -153,18 +144,62 @@ class SettingsEOS(BaseSettings):
     Used by updating the configuration with specific settings only.
     """
 
-    general: Optional[GeneralSettings] = None
-    logging: Optional[LoggingCommonSettings] = None
-    devices: Optional[DevicesCommonSettings] = None
-    measurement: Optional[MeasurementCommonSettings] = None
-    optimization: Optional[OptimizationCommonSettings] = None
-    prediction: Optional[PredictionCommonSettings] = None
-    elecprice: Optional[ElecPriceCommonSettings] = None
-    load: Optional[LoadCommonSettings] = None
-    pvforecast: Optional[PVForecastCommonSettings] = None
-    weather: Optional[WeatherCommonSettings] = None
-    server: Optional[ServerCommonSettings] = None
-    utils: Optional[UtilsCommonSettings] = None
+    general: Optional[GeneralSettings] = Field(
+        default=None,
+        description="General Settings",
+    )
+    cache: Optional[CacheCommonSettings] = Field(
+        default=None,
+        description="Cache Settings",
+    )
+    ems: Optional[EnergyManagementCommonSettings] = Field(
+        default=None,
+        description="Energy Management Settings",
+    )
+    logging: Optional[LoggingCommonSettings] = Field(
+        default=None,
+        description="Logging Settings",
+    )
+    devices: Optional[DevicesCommonSettings] = Field(
+        default=None,
+        description="Devices Settings",
+    )
+    measurement: Optional[MeasurementCommonSettings] = Field(
+        default=None,
+        description="Measurement Settings",
+    )
+    optimization: Optional[OptimizationCommonSettings] = Field(
+        default=None,
+        description="Optimization Settings",
+    )
+    prediction: Optional[PredictionCommonSettings] = Field(
+        default=None,
+        description="Prediction Settings",
+    )
+    elecprice: Optional[ElecPriceCommonSettings] = Field(
+        default=None,
+        description="Electricity Price Settings",
+    )
+    load: Optional[LoadCommonSettings] = Field(
+        default=None,
+        description="Load Settings",
+    )
+    pvforecast: Optional[PVForecastCommonSettings] = Field(
+        default=None,
+        description="PV Forecast Settings",
+    )
+    weather: Optional[WeatherCommonSettings] = Field(
+        default=None,
+        description="Weather Settings",
+    )
+    server: Optional[ServerCommonSettings] = Field(
+        default=None,
+        description="Server Settings",
+    )
+    utils: Optional[UtilsCommonSettings] = Field(
+        default=None,
+        description="Utilities Settings",
+    )
 
     model_config = SettingsConfigDict(
         env_nested_delimiter="__",
@@ -181,6 +216,8 @@ class SettingsEOSDefaults(SettingsEOS):
     """
 
     general: GeneralSettings = GeneralSettings()
+    cache: CacheCommonSettings = CacheCommonSettings()
+    ems: EnergyManagementCommonSettings = EnergyManagementCommonSettings()
     logging: LoggingCommonSettings = LoggingCommonSettings()
     devices: DevicesCommonSettings = DevicesCommonSettings()
     measurement: MeasurementCommonSettings = MeasurementCommonSettings()
@@ -290,7 +327,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
             dotenv_settings,
         ]
 
-        file_settings: Optional[ConfigFileSourceMixin] = None
+        file_settings: Optional[JsonConfigSettingsSource] = None
         config_file, exists = cls._get_config_file_path()
         config_dir = config_file.parent
         if not exists:
@@ -335,13 +372,15 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
         """
         if hasattr(self, "_initialized"):
             return
-        super().__init__(*args, **kwargs)
-        self._create_initial_config_file()
-        self._update_data_folder_path()
+        self._setup(self, *args, **kwargs)
 
     def _setup(self, *args: Any, **kwargs: Any) -> None:
         """Re-initialize global settings."""
+        # Assure settings base knows EOS configuration
+        SettingsBaseModel.config = self
+        # (Re-)load settings
         SettingsEOSDefaults.__init__(self, *args, **kwargs)
+        # Init config file and data folder pathes
         self._create_initial_config_file()
         self._update_data_folder_path()
 
@@ -417,7 +456,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
         if self.general.config_file_path and not self.general.config_file_path.exists():
             self.general.config_file_path.parent.mkdir(parents=True, exist_ok=True)
             try:
-                with open(self.general.config_file_path, "w") as f:
+                with self.general.config_file_path.open("w", encoding="utf-8", newline="\n") as f:
                     f.write(self.model_dump_json(indent=4))
             except Exception as e:
                 logger.error(
@@ -489,7 +528,7 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
         """
         if not self.general.config_file_path:
             raise ValueError("Configuration file path unknown.")
-        with self.general.config_file_path.open("w", encoding=self.ENCODING) as f_out:
+        with self.general.config_file_path.open("w", encoding="utf-8", newline="\n") as f_out:
             json_str = super().model_dump_json()
             f_out.write(json_str)
 

src/akkudoktoreos/config/configabc.py

@@ -1,9 +1,12 @@
 """Abstract and base classes for configuration."""
 
+from typing import Any, ClassVar
+
 from akkudoktoreos.core.pydantic import PydanticBaseModel
 
 
 class SettingsBaseModel(PydanticBaseModel):
     """Base model class for all settings configurations."""
 
-    pass
+    # EOS configuration - set by ConfigEOS
+    config: ClassVar[Any] = None

src/akkudoktoreos/core/cache.py

@@ -1,32 +1,14 @@
-"""Class for in-memory managing of cache files.
+"""In-memory and file caching.
 
-The `CacheFileStore` class is a singleton-based, thread-safe key-value store for managing
-temporary file objects, allowing the creation, retrieval, and management of cache files.
-
-Classes:
---------
-- CacheFileStore: A thread-safe, singleton class for in-memory managing of file-like cache objects.
-- CacheFileStoreMeta: Metaclass for enforcing the singleton behavior in `CacheFileStore`.
-
-Example usage:
---------------
-    # CacheFileStore usage
-    >>> cache_store = CacheFileStore()
-    >>> cache_store.create('example_key')
-    >>> cache_file = cache_store.get('example_key')
-    >>> cache_file.write('Some data')
-    >>> cache_file.seek(0)
-    >>> print(cache_file.read())  # Output: 'Some data'
-
-Notes:
-------
-- Cache files are automatically associated with the current date unless specified.
+Decorators and classes for caching results of computations,
+both in memory (using an LRU cache) and in temporary files. It also includes
+mechanisms for managing cache file expiration and retrieval.
 """
 
-from __future__ import annotations
-
+import functools
 import hashlib
 import inspect
+import json
 import os
 import pickle
 import tempfile
@@ -35,8 +17,8 @@ from typing import (
     IO,
     Any,
     Callable,
+    ClassVar,
     Dict,
-    Generic,
     List,
     Literal,
     Optional,
@@ -44,29 +26,226 @@ from typing import (
     TypeVar,
 )
 
+import cachebox
 from pendulum import DateTime, Duration
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import Field
 
-from akkudoktoreos.core.coreabc import ConfigMixin
+from akkudoktoreos.core.coreabc import ConfigMixin, SingletonMixin
 from akkudoktoreos.core.logging import get_logger
+from akkudoktoreos.core.pydantic import PydanticBaseModel
 from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration
 
 logger = get_logger(__name__)
 
 
-T = TypeVar("T")
+# ---------------------------------
+# In-Memory Caching Functionality
+# ---------------------------------
+
+# Define a type variable for methods and functions
+TCallable = TypeVar("TCallable", bound=Callable[..., Any])
+
+
+def cache_until_update_store_callback(event: int, key: Any, value: Any) -> None:
+    """Calback function for CacheUntilUpdateStore."""
+    CacheUntilUpdateStore.last_event = event
+    CacheUntilUpdateStore.last_key = key
+    CacheUntilUpdateStore.last_value = value
+    if event == cachebox.EVENT_MISS:
+        CacheUntilUpdateStore.miss_count += 1
+    elif event == cachebox.EVENT_HIT:
+        CacheUntilUpdateStore.hit_count += 1
+    else:
+        # unreachable code
+        raise NotImplementedError
+
+
+class CacheUntilUpdateStore(SingletonMixin):
+    """Singleton-based in-memory LRU (Least Recently Used) cache.
+
+    This cache is shared across the application to store results of decorated
+    methods or functions until the next EMS (Energy Management System) update.
+
+    The cache uses an LRU eviction strategy, storing up to 100 items, with the oldest
+    items being evicted once the cache reaches its capacity.
+    """
+
+    cache: ClassVar[cachebox.LRUCache] = cachebox.LRUCache(maxsize=100, iterable=None, capacity=100)
+    last_event: ClassVar[Optional[int]] = None
+    last_key: ClassVar[Any] = None
+    last_value: ClassVar[Any] = None
+    hit_count: ClassVar[int] = 0
+    miss_count: ClassVar[int] = 0
+
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
+        """Initializes the `CacheUntilUpdateStore` instance with default parameters.
+
+        The cache uses an LRU eviction strategy with a maximum size of 100 items.
+        This cache is a singleton, meaning only one instance will exist throughout
+        the application lifecycle.
+
+        Example:
+            >>> cache = CacheUntilUpdateStore()
+        """
+        if hasattr(self, "_initialized"):
+            return
+        super().__init__(*args, **kwargs)
+
+    def __getattr__(self, name: str) -> Any:
+        """Propagates method calls to the cache object.
+
+        This method allows you to call methods on the underlying cache object,
+        and it will delegate the call to the cache's corresponding method.
+
+        Args:
+            name (str): The name of the method being called.
+
+        Returns:
+            Callable: A method bound to the cache object.
+
+        Raises:
+            AttributeError: If the cache object does not have the requested method.
+
+        Example:
+            >>> result = cache.get("key")
+        """
+        # This will return a method of the target cache, or raise an AttributeError
+        target_attr = getattr(self.cache, name)
+        if callable(target_attr):
+            return target_attr
+        else:
+            return target_attr
+
+    def __getitem__(self, key: Any) -> Any:
+        """Retrieves an item from the cache by its key.
+
+        Args:
+            key (Any): The key used for subscripting to retrieve an item.
+
+        Returns:
+            Any: The value corresponding to the key in the cache.
+
+        Raises:
+            KeyError: If the key does not exist in the cache.
+
+        Example:
+            >>> value = cache["user_data"]
+        """
+        return CacheUntilUpdateStore.cache[key]
+
+    def __setitem__(self, key: Any, value: Any) -> None:
+        """Stores an item in the cache.
+
+        Args:
+            key (Any): The key used to store the item in the cache.
+            value (Any): The value to store.
+
+        Example:
+            >>> cache["user_data"] = {"name": "Alice", "age": 30}
+        """
+        CacheUntilUpdateStore.cache[key] = value
+
+    def __len__(self) -> int:
+        """Returns the number of items in the cache."""
+        return len(CacheUntilUpdateStore.cache)
+
+    def __repr__(self) -> str:
+        """Provides a string representation of the CacheUntilUpdateStore object."""
+        return repr(CacheUntilUpdateStore.cache)
+
+    def clear(self) -> None:
+        """Clears the cache, removing all stored items.
+
+        This method propagates the `clear` method call to the underlying cache object,
+        ensuring that the cache is emptied when necessary (e.g., at the end of the energy
+        management system run).
+
+        Example:
+            >>> cache.clear()
+        """
+        if hasattr(self.cache, "clear") and callable(getattr(self.cache, "clear")):
+            CacheUntilUpdateStore.cache.clear()
+            CacheUntilUpdateStore.last_event = None
+            CacheUntilUpdateStore.last_key = None
+            CacheUntilUpdateStore.last_value = None
+            CacheUntilUpdateStore.miss_count = 0
+            CacheUntilUpdateStore.hit_count = 0
+        else:
+            raise AttributeError(f"'{self.cache.__class__.__name__}' object has no method 'clear'")
+
+
+def cachemethod_until_update(method: TCallable) -> TCallable:
+    """Decorator for in memory caching the result of an instance method.
+
+    This decorator caches the method's result in `CacheUntilUpdateStore`, ensuring
+    that subsequent calls with the same arguments return the cached result until the
+    next EMS update cycle.
+
+    Args:
+        method (Callable): The instance method to be decorated.
+
+    Returns:
+        Callable: The wrapped method with caching functionality.
+
+    Example:
+        >>> class MyClass:
+        >>>     @cachemethod_until_update
+        >>>     def expensive_method(self, param: str) -> str:
+        >>>         # Perform expensive computation
+        >>>         return f"Computed {param}"
+    """
+
+    @cachebox.cachedmethod(
+        cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback
+    )
+    @functools.wraps(method)
+    def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
+        result = method(self, *args, **kwargs)
+        return result
+
+    return wrapper
+
+
+def cache_until_update(func: TCallable) -> TCallable:
+    """Decorator for in memory caching the result of a standalone function.
+
+    This decorator caches the function's result in `CacheUntilUpdateStore`, ensuring
+    that subsequent calls with the same arguments return the cached result until the
+    next EMS update cycle.
+
+    Args:
+        func (Callable): The function to be decorated.
+
+    Returns:
+        Callable: The wrapped function with caching functionality.
+
+    Example:
+        >>> @cache_until_next_update
+        >>> def expensive_function(param: str) -> str:
+        >>>     # Perform expensive computation
+        >>>     return f"Computed {param}"
+    """
+
+    @cachebox.cached(
+        cache=CacheUntilUpdateStore().cache, callback=cache_until_update_store_callback
+    )
+    @functools.wraps(func)
+    def wrapper(*args: Any, **kwargs: Any) -> Any:
+        result = func(*args, **kwargs)
+        return result
+
+    return wrapper
+
+
+# ---------------------------------
+# Cache File Management
+# ---------------------------------
+
 Param = ParamSpec("Param")
 RetType = TypeVar("RetType")
 
 
-class CacheFileRecord(BaseModel):
-    # Enable custom serialization globally in config
-    model_config = ConfigDict(
-        arbitrary_types_allowed=True,
-        use_enum_values=True,
-        validate_assignment=True,
-    )
-
+class CacheFileRecord(PydanticBaseModel):
     cache_file: Any = Field(..., description="File descriptor of the cache file.")
     until_datetime: DateTime = Field(..., description="Datetime until the cache file is valid.")
     ttl_duration: Optional[Duration] = Field(
@@ -74,24 +253,7 @@ class CacheFileRecord(BaseModel):
     )
 
 
-class CacheFileStoreMeta(type, Generic[T]):
-    """A thread-safe implementation of CacheFileStore."""
-
-    _instances: dict[CacheFileStoreMeta[T], T] = {}
-
-    _lock: threading.Lock = threading.Lock()
-    """Lock object to synchronize threads on first access to CacheFileStore."""
-
-    def __call__(cls) -> T:
-        """Return CacheFileStore instance."""
-        with cls._lock:
-            if cls not in cls._instances:
-                instance = super().__call__()
-                cls._instances[cls] = instance
-        return cls._instances[cls]
-
-
-class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
+class CacheFileStore(ConfigMixin, SingletonMixin):
     """A key-value store that manages file-like tempfile objects to be used as cache files.
 
     Cache files are associated with a date. If no date is specified, the cache files are
@@ -105,7 +267,7 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
         store (dict): A dictionary that holds the in-memory cache file objects
                       with their associated keys and dates.
 
-    Example usage:
+    Example:
         >>> cache_store = CacheFileStore()
         >>> cache_store.create('example_file')
         >>> cache_file = cache_store.get('example_file')
@@ -114,14 +276,18 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
         >>> print(cache_file.read())  # Output: 'Some data'
     """
 
-    def __init__(self) -> None:
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
         """Initializes the CacheFileStore instance.
 
         This constructor sets up an empty key-value store (a dictionary) where each key
         corresponds to a cache file that is associated with a given key and an optional date.
         """
+        if hasattr(self, "_initialized"):
+            return
         self._store: Dict[str, CacheFileRecord] = {}
-        self._store_lock = threading.Lock()
+        self._store_lock = threading.RLock()
+        self._store_file = self.config.cache.path().joinpath("cachefilestore.json")
+        super().__init__(*args, **kwargs)
 
     def _until_datetime_by_options(
         self,
@@ -329,9 +495,9 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
                 # File already available
                 cache_file_obj = cache_item.cache_file
             else:
-                self.config.general.data_cache_path.mkdir(parents=True, exist_ok=True)
+                self.config.cache.path().mkdir(parents=True, exist_ok=True)
                 cache_file_obj = tempfile.NamedTemporaryFile(
-                    mode=mode, delete=delete, suffix=suffix, dir=self.config.general.data_cache_path
+                    mode=mode, delete=delete, suffix=suffix, dir=self.config.cache.path()
                 )
                 self._store[cache_file_key] = CacheFileRecord(
                     cache_file=cache_file_obj,
@@ -502,7 +668,7 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
 
     def clear(
         self,
-        clear_all: bool = False,
+        clear_all: Optional[bool] = None,
         before_datetime: Optional[Any] = None,
     ) -> None:
         """Deletes all cache files or those expiring before `before_datetime`.
@@ -516,8 +682,6 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
         Raises:
             OSError: If there's an error during file deletion.
         """
-        delete_keys = []  # List of keys to delete, prevent deleting when traversing the store
-
         # Some weired logic to prevent calling to_datetime on clear_all.
         # Clear_all may be set on __del__. At this time some info for to_datetime will
         # not be available anymore.
@@ -528,6 +692,8 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
                 before_datetime = to_datetime(before_datetime)
 
         with self._store_lock:  # Synchronize access to _store
+            delete_keys = []  # List of keys to delete, prevent deleting when traversing the store
+
             for cache_file_key, cache_item in self._store.items():
                 # Some weired logic to prevent calling to_datetime on clear_all.
                 # Clear_all may be set on __del__. At this time some info for to_datetime will
@@ -566,6 +732,89 @@ class CacheFileStore(ConfigMixin, metaclass=CacheFileStoreMeta):
             for delete_key in delete_keys:
                 del self._store[delete_key]
 
+    def current_store(self) -> dict:
+        """Current state of the store.
+
+        Returns:
+            data (dict): current cache management data.
+        """
+        with self._store_lock:
+            store_current = {}
+            for key, record in self._store.items():
+                ttl_duration = record.ttl_duration
+                if ttl_duration:
+                    ttl_duration = ttl_duration.total_seconds()
+                store_current[key] = {
+                    # Convert file-like objects to file paths for serialization
+                    "cache_file": self._get_file_path(record.cache_file),
+                    "mode": record.cache_file.mode,
+                    "until_datetime": to_datetime(record.until_datetime, as_string=True),
+                    "ttl_duration": ttl_duration,
+                }
+            return store_current
+
+    def save_store(self) -> dict:
+        """Saves the current state of the store to a file.
+
+        Returns:
+            data (dict): cache management data that was saved.
+        """
+        with self._store_lock:
+            self._store_file.parent.mkdir(parents=True, exist_ok=True)
+            store_to_save = self.current_store()
+            with self._store_file.open("w", encoding="utf-8", newline="\n") as f:
+                try:
+                    json.dump(store_to_save, f, indent=4)
+                except Exception as e:
+                    logger.error(f"Error saving cache file store: {e}")
+            return store_to_save
+
+    def load_store(self) -> dict:
+        """Loads the state of the store from a file.
+
+        Returns:
+            data (dict): cache management data that was loaded.
+        """
+        with self._store_lock:
+            store_loaded = {}
+            if self._store_file.exists():
+                with self._store_file.open("r", encoding="utf-8", newline=None) as f:
+                    try:
+                        store_to_load = json.load(f)
+                    except Exception as e:
+                        logger.error(
+                            f"Error loading cache file store: {e}\n"
+                            + f"Deleting the store file {self._store_file}."
+                        )
+                        self._store_file.unlink()
+                        return {}
+                    for key, record in store_to_load.items():
+                        if record is None:
+                            continue
+                        if key in self._store.keys():
+                            # Already available - do not overwrite by record from file
+                            continue
+                        try:
+                            cache_file_obj = open(
+                                record["cache_file"], "rb+" if "b" in record["mode"] else "r+"
+                            )
+                        except Exception as e:
+                            cache_file_record = record["cache_file"]
+                            logger.warning(f"Can not open cache file '{cache_file_record}': {e}")
+                            continue
+                        ttl_duration = record["ttl_duration"]
+                        if ttl_duration:
+                            ttl_duration = to_duration(float(record["ttl_duration"]))
+                        self._store[key] = CacheFileRecord(
+                            cache_file=cache_file_obj,
+                            until_datetime=record["until_datetime"],
+                            ttl_duration=ttl_duration,
+                        )
+                        cache_file_obj.seek(0)
+                        # Remember newly loaded
+                        store_loaded[key] = record
+            return store_loaded
+
 
 def cache_in_file(
     ignore_params: List[str] = [],

src/akkudoktoreos/core/cachesettings.py

@@ -0,0 +1,32 @@
+"""Settings for caching.
+
+Kept in an extra module to avoid cyclic dependencies on package import.
+"""
+
+from pathlib import Path
+from typing import Optional
+
+from pydantic import Field
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+
+
+class CacheCommonSettings(SettingsBaseModel):
+    """Cache Configuration."""
+
+    subpath: Optional[Path] = Field(
+        default="cache", description="Sub-path for the EOS cache data directory."
+    )
+
+    cleanup_interval: float = Field(
+        default=5 * 60, description="Intervall in seconds for EOS file cache cleanup."
+    )
+
+    # Do not make this a pydantic computed field. The pydantic model must be fully initialized
+    # to have access to config.general, which may not be the case if it is a computed field.
+    def path(self) -> Optional[Path]:
+        """Compute cache path based on general.data_folder_path."""
+        data_cache_path = self.config.general.data_folder_path
+        if data_cache_path is None or self.subpath is None:
+            return None
+        return data_cache_path.joinpath(self.subpath)

src/akkudoktoreos/core/coreabc.py

@@ -265,10 +265,12 @@ class SingletonMixin:
         class MySingletonModel(SingletonMixin, PydanticBaseModel):
             name: str
 
-            # implement __init__ to avoid re-initialization of parent class PydanticBaseModel:
+            # implement __init__ to avoid re-initialization of parent classes:
             def __init__(self, *args: Any, **kwargs: Any) -> None:
                 if hasattr(self, "_initialized"):
                     return
+                # Your initialisation here
+                ...
                 super().__init__(*args, **kwargs)
 
         instance1 = MySingletonModel(name="Instance 1")

src/akkudoktoreos/core/dataabc.py

@@ -953,6 +953,44 @@ class DataSequence(DataBase, MutableSequence):
         array = resampled.values
         return array
 
+    def to_dataframe(
+        self,
+        start_datetime: Optional[DateTime] = None,
+        end_datetime: Optional[DateTime] = None,
+    ) -> pd.DataFrame:
+        """Converts the sequence of DataRecord instances into a Pandas DataFrame.
+
+        Args:
+            start_datetime (Optional[datetime]): The lower bound for filtering (inclusive).
+                Defaults to the earliest possible datetime if None.
+            end_datetime (Optional[datetime]): The upper bound for filtering (exclusive).
+                Defaults to the latest possible datetime if None.
+
+        Returns:
+            pd.DataFrame: A DataFrame containing the filtered data from all records.
+        """
+        if not self.records:
+            return pd.DataFrame()  # Return empty DataFrame if no records exist
+
+        # Use filter_by_datetime to get filtered records
+        filtered_records = self.filter_by_datetime(start_datetime, end_datetime)
+
+        # Convert filtered records to a dictionary list
+        data = [record.model_dump() for record in filtered_records]
+
+        # Convert to DataFrame
+        df = pd.DataFrame(data)
+        if df.empty:
+            return df
+
+        # Ensure `date_time` column exists and use it for the index
+        if not "date_time" in df.columns:
+            error_msg = f"Cannot create dataframe: no `date_time` column in `{df}`."
+            logger.error(error_msg)
+            raise TypeError(error_msg)
+        df.index = pd.DatetimeIndex(df["date_time"])
+        return df
+
     def sort_by_datetime(self, reverse: bool = False) -> None:
         """Sort the DataRecords in the sequence by their date_time attribute.
 
@@ -1465,7 +1503,7 @@ class DataImportMixin:
                 error_msg += f"Field: {field}\nError: {message}\nType: {error_type}\n"
             logger.debug(f"PydanticDateTimeDataFrame import: {error_msg}")
 
-        # Try dictionary with special keys start_datetime and intervall
+        # Try dictionary with special keys start_datetime and interval
         try:
             import_data = PydanticDateTimeData.model_validate_json(json_str)
             self.import_from_dict(import_data.to_dict())
@@ -1525,7 +1563,7 @@ class DataImportMixin:
             and `key_prefix = "load"`, only the "load_mean" key will be processed even though
             both keys are in the record.
         """
-        with import_file_path.open("r") as import_file:
+        with import_file_path.open("r", encoding="utf-8", newline=None) as import_file:
             import_str = import_file.read()
         self.import_from_json(
             import_str, key_prefix=key_prefix, start_datetime=start_datetime, interval=interval

src/akkudoktoreos/core/ems.py

@@ -6,19 +6,20 @@ from pendulum import DateTime
 from pydantic import ConfigDict, Field, computed_field, field_validator, model_validator
 from typing_extensions import Self
 
+from akkudoktoreos.core.cache import CacheUntilUpdateStore
 from akkudoktoreos.core.coreabc import ConfigMixin, PredictionMixin, SingletonMixin
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.pydantic import ParametersBaseModel, PydanticBaseModel
 from akkudoktoreos.devices.battery import Battery
 from akkudoktoreos.devices.generic import HomeAppliance
 from akkudoktoreos.devices.inverter import Inverter
-from akkudoktoreos.utils.datetimeutil import to_datetime
+from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
 from akkudoktoreos.utils.utils import NumpyEncoder
 
 logger = get_logger(__name__)
 
 
-class EnergieManagementSystemParameters(ParametersBaseModel):
+class EnergyManagementParameters(ParametersBaseModel):
     pv_prognose_wh: list[float] = Field(
         description="An array of floats representing the forecasted photovoltaic output in watts for different time intervals."
     )
@@ -107,7 +108,7 @@ class SimulationResult(ParametersBaseModel):
         return NumpyEncoder.convert_numpy(field)[0]
 
 
-class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBaseModel):
+class EnergyManagement(SingletonMixin, ConfigMixin, PredictionMixin, PydanticBaseModel):
     # Disable validation on assignment to speed up simulation runs.
     model_config = ConfigDict(
         validate_assignment=False,
@@ -116,16 +117,33 @@ class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, Pyda
     # Start datetime.
     _start_datetime: ClassVar[Optional[DateTime]] = None
 
+    # last run datetime. Used by energy management task
+    _last_datetime: ClassVar[Optional[DateTime]] = None
+
     @computed_field  # type: ignore[prop-decorator]
     @property
     def start_datetime(self) -> DateTime:
         """The starting datetime of the current or latest energy management."""
-        if EnergieManagementSystem._start_datetime is None:
-            EnergieManagementSystem.set_start_datetime()
-        return EnergieManagementSystem._start_datetime
+        if EnergyManagement._start_datetime is None:
+            EnergyManagement.set_start_datetime()
+        return EnergyManagement._start_datetime
 
     @classmethod
     def set_start_datetime(cls, start_datetime: Optional[DateTime] = None) -> DateTime:
+        """Set the start datetime for the next energy management cycle.
+
+        If no datetime is provided, the current datetime is used.
+
+        The start datetime is always rounded down to the nearest hour
+        (i.e., setting minutes, seconds, and microseconds to zero).
+
+        Args:
+            start_datetime (Optional[DateTime]): The datetime to set as the start.
+                If None, the current datetime is used.
+
+        Returns:
+            DateTime: The adjusted start datetime.
+        """
         if start_datetime is None:
             start_datetime = to_datetime()
         cls._start_datetime = start_datetime.set(minute=0, second=0, microsecond=0)
@@ -176,7 +194,7 @@ class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, Pyda
 
     def set_parameters(
         self,
-        parameters: EnergieManagementSystemParameters,
+        parameters: EnergyManagementParameters,
         ev: Optional[Battery] = None,
         home_appliance: Optional[HomeAppliance] = None,
         inverter: Optional[Inverter] = None,
@@ -243,6 +261,8 @@ class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, Pyda
             is mostly relevant to prediction providers.
             force_update (bool, optional): If True, forces to update the data even if still cached.
         """
+        # Throw away any cached results of the last run.
+        CacheUntilUpdateStore().clear()
         self.set_start_hour(start_hour=start_hour)
 
         # Check for run definitions
@@ -254,14 +274,70 @@ class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, Pyda
             error_msg = "Prediction hours unknown."
             logger.error(error_msg)
             raise ValueError(error_msg)
-        if self.config.prediction.optimisation_hours is None:
-            error_msg = "Optimisation hours unknown."
+        if self.config.optimization.hours is None:
+            error_msg = "Optimization hours unknown."
             logger.error(error_msg)
             raise ValueError(error_msg)
 
         self.prediction.update_data(force_enable=force_enable, force_update=force_update)
         # TODO: Create optimisation problem that calls into devices.update_data() for simulations.
 
+    def manage_energy(self) -> None:
+        """Repeating task for managing energy.
+
+        This task should be executed by the server regularly (e.g., every 10 seconds)
+        to ensure proper energy management. Configuration changes to the energy management interval
+        will only take effect if this task is executed.
+
+        - Initializes and runs the energy management for the first time if it has never been run
+          before.
+        - If the energy management interval is not configured or invalid (NaN), the task will not
+          trigger any repeated energy management runs.
+        - Compares the current time with the last run time and runs the energy management if the
+          interval has elapsed.
+        - Logs any exceptions that occur during the initialization or execution of the energy
+          management.
+
+        Note: The task maintains the interval even if some intervals are missed.
+        """
+        current_datetime = to_datetime()
+
+        if EnergyManagement._last_datetime is None:
+            # Never run before
+            try:
+                # Try to run a first energy management. May fail due to config incomplete.
+                self.run()
+                # Remember energy run datetime.
+                EnergyManagement._last_datetime = current_datetime
+            except Exception as e:
+                message = f"EOS init: {e}"
+                logger.error(message)
+            return
+
+        if self.config.ems.interval is None or self.config.ems.interval == float("nan"):
+            # No Repetition
+            return
+
+        if (
+            compare_datetimes(current_datetime, self._last_datetime).time_diff
+            < self.config.ems.interval
+        ):
+            # Wait for next run
+            return
+
+        try:
+            self.run()
+        except Exception as e:
+            message = f"EOS run: {e}"
+            logger.error(message)
+
+        # Remember the energy management run - keep on interval even if we missed some intervals
+        while (
+            compare_datetimes(current_datetime, EnergyManagement._last_datetime).time_diff
+            >= self.config.ems.interval
+        ):
+            EnergyManagement._last_datetime.add(seconds=self.config.ems.interval)
+
     def set_start_hour(self, start_hour: Optional[int] = None) -> None:
         """Sets start datetime to given hour.
 
@@ -439,9 +515,9 @@ class EnergieManagementSystem(SingletonMixin, ConfigMixin, PredictionMixin, Pyda
 
 
 # Initialize the Energy Management System, it is a singleton.
-ems = EnergieManagementSystem()
+ems = EnergyManagement()
 
 
-def get_ems() -> EnergieManagementSystem:
+def get_ems() -> EnergyManagement:
     """Gets the EOS Energy Management System."""
     return ems

src/akkudoktoreos/core/emsettings.py

@@ -0,0 +1,26 @@
+"""Settings for energy management.
+
+Kept in an extra module to avoid cyclic dependencies on package import.
+"""
+
+from typing import Optional
+
+from pydantic import Field
+
+from akkudoktoreos.config.configabc import SettingsBaseModel
+
+
+class EnergyManagementCommonSettings(SettingsBaseModel):
+    """Energy Management Configuration."""
+
+    startup_delay: float = Field(
+        default=5,
+        ge=1,
+        description="Startup delay in seconds for EOS energy management runs.",
+    )
+
+    interval: Optional[float] = Field(
+        default=None,
+        description="Intervall in seconds between EOS energy management runs.",
+        examples=["300"],
+    )

src/akkudoktoreos/optimization/genetic.py

@@ -12,7 +12,7 @@ from akkudoktoreos.core.coreabc import (
     DevicesMixin,
     EnergyManagementSystemMixin,
 )
-from akkudoktoreos.core.ems import EnergieManagementSystemParameters, SimulationResult
+from akkudoktoreos.core.ems import EnergyManagementParameters, SimulationResult
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.pydantic import ParametersBaseModel
 from akkudoktoreos.devices.battery import (
@@ -29,7 +29,7 @@ logger = get_logger(__name__)
 
 
 class OptimizationParameters(ParametersBaseModel):
-    ems: EnergieManagementSystemParameters
+    ems: EnergyManagementParameters
     pv_akku: Optional[SolarPanelBatteryParameters]
     inverter: Optional[InverterParameters]
     eauto: Optional[ElectricVehicleParameters]

src/akkudoktoreos/prediction/elecpriceakkudoktor.py

@@ -14,10 +14,10 @@ import requests
 from pydantic import ValidationError
 from statsmodels.tsa.holtwinters import ExponentialSmoothing
 
+from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.pydantic import PydanticBaseModel
 from akkudoktoreos.prediction.elecpriceabc import ElecPriceProvider
-from akkudoktoreos.utils.cacheutil import cache_in_file
 from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
 
 logger = get_logger(__name__)

src/akkudoktoreos/prediction/pvforecastakkudoktor.py

@@ -80,13 +80,13 @@ from typing import Any, List, Optional, Union
 import requests
 from pydantic import Field, ValidationError, computed_field
 
+from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.pydantic import PydanticBaseModel
 from akkudoktoreos.prediction.pvforecastabc import (
     PVForecastDataRecord,
     PVForecastProvider,
 )
-from akkudoktoreos.utils.cacheutil import cache_in_file
 from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
 
 logger = get_logger(__name__)
@@ -267,7 +267,7 @@ class PVForecastAkkudoktor(PVForecastProvider):
         logger.debug(f"Response from {self._url()}: {response}")
         akkudoktor_data = self._validate_data(response.content)
         # We are working on fresh data (no cache), report update time
-        self.update_datetime = to_datetime(in_timezone=self.config.general.timezone)
+
         return akkudoktor_data
 
     def _update_data(self, force_update: Optional[bool] = False) -> None:

src/akkudoktoreos/prediction/weatherbrightsky.py

@@ -13,9 +13,9 @@ import pandas as pd
 import pvlib
 import requests
 
+from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.prediction.weatherabc import WeatherDataRecord, WeatherProvider
-from akkudoktoreos.utils.cacheutil import cache_in_file
 from akkudoktoreos.utils.datetimeutil import to_datetime
 
 logger = get_logger(__name__)

src/akkudoktoreos/prediction/weatherclearoutside.py

@@ -19,9 +19,9 @@ import pandas as pd
 import requests
 from bs4 import BeautifulSoup
 
+from akkudoktoreos.core.cache import cache_in_file
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.prediction.weatherabc import WeatherDataRecord, WeatherProvider
-from akkudoktoreos.utils.cacheutil import cache_in_file
 from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration, to_timezone
 
 logger = get_logger(__name__)

src/akkudoktoreos/server/eos.py

@@ -1,22 +1,34 @@
 #!/usr/bin/env python3
 
 import argparse
+import asyncio
+import json
 import os
+import signal
 import subprocess
 import sys
+import time
 from contextlib import asynccontextmanager
 from pathlib import Path
 from typing import Annotated, Any, AsyncGenerator, Dict, List, Optional, Union
 
 import httpx
+import psutil
 import uvicorn
 from fastapi import Body, FastAPI
 from fastapi import Path as FastapiPath
 from fastapi import Query, Request
 from fastapi.exceptions import HTTPException
-from fastapi.responses import FileResponse, HTMLResponse, RedirectResponse, Response
+from fastapi.responses import (
+    FileResponse,
+    HTMLResponse,
+    JSONResponse,
+    RedirectResponse,
+    Response,
+)
 
 from akkudoktoreos.config.config import ConfigEOS, SettingsEOS, get_config
+from akkudoktoreos.core.cache import CacheFileStore
 from akkudoktoreos.core.ems import get_ems
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.core.pydantic import (
@@ -36,6 +48,8 @@ from akkudoktoreos.prediction.load import LoadCommonSettings
 from akkudoktoreos.prediction.loadakkudoktor import LoadAkkudoktorCommonSettings
 from akkudoktoreos.prediction.prediction import PredictionCommonSettings, get_prediction
 from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings
+from akkudoktoreos.server.rest.tasks import repeat_every
+from akkudoktoreos.server.server import get_default_host
 from akkudoktoreos.utils.datetimeutil import to_datetime, to_duration
 
 logger = get_logger(__name__)
@@ -145,35 +159,58 @@ def create_error_page(
 # ----------------------
 
 
-def start_eosdash() -> subprocess.Popen:
+def start_eosdash(
+    host: str,
+    port: int,
+    eos_host: str,
+    eos_port: int,
+    log_level: str,
+    access_log: bool,
+    reload: bool,
+    eos_dir: str,
+    eos_config_dir: str,
+) -> subprocess.Popen:
     """Start the EOSdash server as a subprocess.
 
+    This function starts the EOSdash server by launching it as a subprocess. It checks if the server
+    is already running on the specified port and either returns the existing process or starts a new one.
+
+    Args:
+        host (str): The hostname for the EOSdash server.
+        port (int): The port for the EOSdash server.
+        eos_host (str): The hostname for the EOS server.
+        eos_port (int): The port for the EOS server.
+        log_level (str): The logging level for the EOSdash server.
+        access_log (bool): Flag to enable or disable access logging.
+        reload (bool): Flag to enable or disable auto-reloading.
+        eos_dir (str): Path to the EOS data directory.
+        eos_config_dir (str): Path to the EOS configuration directory.
+
     Returns:
-        server_process: The process of the EOSdash server
+        subprocess.Popen: The process of the EOSdash server.
+
+    Raises:
+        RuntimeError: If the EOSdash server fails to start.
     """
     eosdash_path = Path(__file__).parent.resolve().joinpath("eosdash.py")
 
-    if args is None:
-        # No command line arguments
-        host = config_eos.server.eosdash_host
-        port = config_eos.server.eosdash_port
-        eos_host = config_eos.server.host
-        eos_port = config_eos.server.port
-        log_level = "info"
-        access_log = False
-        reload = False
-    else:
-        host = args.host
-        port = config_eos.server.eosdash_port if config_eos.server.eosdash_port else (args.port + 1)
-        eos_host = args.host
-        eos_port = args.port
-        log_level = args.log_level
-        access_log = args.access_log
-        reload = args.reload
+    # Check if the EOSdash process is still/ already running, e.g. in case of server restart
+    process_info = None
+    for conn in psutil.net_connections(kind="inet"):
+        if conn.laddr.port == port:
+            process = psutil.Process(conn.pid)
+            # Get the fresh process info
+            process_info = process.as_dict(attrs=["pid", "cmdline"])
+            break
+    if process_info:
+        # Just warn
+        logger.warning(f"EOSdash port `{port}` still/ already in use.")
+        logger.warning(f"PID: `{process_info['pid']}`, CMD: `{process_info['cmdline']}`")
 
     cmd = [
         sys.executable,
-        str(eosdash_path),
+        "-m",
+        "akkudoktoreos.server.eosdash",
         "--host",
         str(host),
         "--port",
@@ -189,11 +226,23 @@ def start_eosdash() -> subprocess.Popen:
         "--reload",
         str(reload),
     ]
-    server_process = subprocess.Popen(
-        cmd,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE,
-    )
+    # Set environment before any subprocess run, to keep custom config dir
+    env = os.environ.copy()
+    env["EOS_DIR"] = eos_dir
+    env["EOS_CONFIG_DIR"] = eos_config_dir
+
+    try:
+        server_process = subprocess.Popen(
+            cmd,
+            env=env,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            start_new_session=True,
+        )
+    except subprocess.CalledProcessError as ex:
+        error_msg = f"Could not start EOSdash: {ex}"
+        logger.error(error_msg)
+        raise RuntimeError(error_msg)
 
     return server_process
 
@@ -203,20 +252,130 @@ def start_eosdash() -> subprocess.Popen:
 # ----------------------
 
 
+def cache_clear(clear_all: Optional[bool] = None) -> None:
+    """Cleanup expired cache files."""
+    if clear_all:
+        CacheFileStore().clear(clear_all=True)
+    else:
+        CacheFileStore().clear(before_datetime=to_datetime())
+
+
+def cache_load() -> dict:
+    """Load cache from cachefilestore.json."""
+    return CacheFileStore().load_store()
+
+
+def cache_save() -> dict:
+    """Save cache to cachefilestore.json."""
+    return CacheFileStore().save_store()
+
+
+@repeat_every(seconds=float(config_eos.cache.cleanup_interval))
+def cache_cleanup_task() -> None:
+    """Repeating task to clear cache from expired cache files."""
+    cache_clear()
+
+
+@repeat_every(
+    seconds=10,
+    wait_first=config_eos.ems.startup_delay,
+)
+def energy_management_task() -> None:
+    """Repeating task for energy management."""
+    ems_eos.manage_energy()
+
+
+async def server_shutdown_task() -> None:
+    """One-shot task for shutting down the EOS server.
+
+    This coroutine performs the following actions:
+    1. Ensures the cache is saved by calling the cache_save function.
+    2. Waits for 5 seconds to allow the EOS server to complete any ongoing tasks.
+    3. Gracefully shuts down the current process by sending the appropriate signal.
+
+    If running on Windows, the CTRL_C_EVENT signal is sent to terminate the process.
+    On other operating systems, the SIGTERM signal is used.
+
+    Finally, logs a message indicating that the EOS server has been terminated.
+    """
+    # Assure cache is saved
+    cache_save()
+
+    # Give EOS time to finish some work
+    await asyncio.sleep(5)
+
+    # Gracefully shut down this process.
+    pid = psutil.Process().pid
+    if os.name == "nt":
+        os.kill(pid, signal.CTRL_C_EVENT)  # type: ignore[attr-defined]
+    else:
+        os.kill(pid, signal.SIGTERM)
+
+    logger.info(f"🚀 EOS terminated, PID {pid}")
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
     """Lifespan manager for the app."""
     # On startup
     if config_eos.server.startup_eosdash:
         try:
-            eosdash_process = start_eosdash()
+            if args is None:
+                # No command line arguments
+                host = config_eos.server.eosdash_host
+                port = config_eos.server.eosdash_port
+                eos_host = config_eos.server.host
+                eos_port = config_eos.server.port
+                log_level = "info"
+                access_log = False
+                reload = False
+            else:
+                host = args.host
+                port = (
+                    config_eos.server.eosdash_port
+                    if config_eos.server.eosdash_port
+                    else (args.port + 1)
+                )
+                eos_host = args.host
+                eos_port = args.port
+                log_level = args.log_level
+                access_log = args.access_log
+                reload = args.reload
+
+            host = host if host else get_default_host()
+            port = port if port else 8504
+            eos_host = eos_host if eos_host else get_default_host()
+            eos_port = eos_port if eos_port else 8503
+
+            eos_dir = str(config_eos.general.data_folder_path)
+            eos_config_dir = str(config_eos.general.config_folder_path)
+
+            eosdash_process = start_eosdash(
+                host=host,
+                port=port,
+                eos_host=eos_host,
+                eos_port=eos_port,
+                log_level=log_level,
+                access_log=access_log,
+                reload=reload,
+                eos_dir=eos_dir,
+                eos_config_dir=eos_config_dir,
+            )
         except Exception as e:
             logger.error(f"Failed to start EOSdash server. Error: {e}")
             sys.exit(1)
+    cache_load()
+    if config_eos.cache.cleanup_interval is None:
+        logger.warning("Cache file cleanup disabled. Set cache.cleanup_interval.")
+    else:
+        await cache_cleanup_task()
+    await energy_management_task()
+
     # Handover to application
     yield
+
     # On shutdown
-    # nothing to do
+    cache_save()
 
 
 app = FastAPI(
@@ -229,9 +388,9 @@ app = FastAPI(
         "url": "https://www.apache.org/licenses/LICENSE-2.0.html",
     },
     lifespan=lifespan,
-    root_path=str(Path(__file__).parent),
 )
 
+
 server_dir = Path(__file__).parent.resolve()
 
 
@@ -239,9 +398,132 @@ class PdfResponse(FileResponse):
     media_type = "application/pdf"
 
 
-@app.put("/v1/config/reset", tags=["config"])
-def fastapi_config_update_post() -> ConfigEOS:
-    """Reset the configuration to the EOS configuration file.
+@app.post("/v1/admin/cache/clear", tags=["admin"])
+def fastapi_admin_cache_clear_post(clear_all: Optional[bool] = None) -> dict:
+    """Clear the cache from expired data.
+
+    Deletes expired cache files.
+
+    Args:
+        clear_all (Optional[bool]): Delete all cached files. Default is False.
+
+    Returns:
+        data (dict): The management data after cleanup.
+    """
+    try:
+        cache_clear(clear_all=clear_all)
+        data = CacheFileStore().current_store()
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=f"Error on cache clear: {e}")
+    return data
+
+
+@app.post("/v1/admin/cache/save", tags=["admin"])
+def fastapi_admin_cache_save_post() -> dict:
+    """Save the current cache management data.
+
+    Returns:
+        data (dict): The management data that was saved.
+    """
+    try:
+        data = cache_save()
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=f"Error on cache save: {e}")
+    return data
+
+
+@app.post("/v1/admin/cache/load", tags=["admin"])
+def fastapi_admin_cache_load_post() -> dict:
+    """Load cache management data.
+
+    Returns:
+        data (dict): The management data that was loaded.
+    """
+    try:
+        data = cache_save()
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=f"Error on cache load: {e}")
+    return data
+
+
+@app.get("/v1/admin/cache", tags=["admin"])
+def fastapi_admin_cache_get() -> dict:
+    """Current cache management data.
+
+    Returns:
+        data (dict): The management data.
+    """
+    try:
+        data = CacheFileStore().current_store()
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=f"Error on cache data retrieval: {e}")
+    return data
+
+
+@app.post("/v1/admin/server/restart", tags=["admin"])
+async def fastapi_admin_server_restart_post() -> dict:
+    """Restart the server.
+
+    Restart EOS properly by starting a new instance before exiting the old one.
+    """
+    logger.info("🔄 Restarting EOS...")
+
+    # Start a new EOS (Uvicorn) process
+    # Force a new process group to make the new process easily distinguishable from the current one
+    # Set environment before any subprocess run, to keep custom config dir
+    env = os.environ.copy()
+    env["EOS_DIR"] = str(config_eos.general.data_folder_path)
+    env["EOS_CONFIG_DIR"] = str(config_eos.general.config_folder_path)
+
+    new_process = subprocess.Popen(
+        [
+            sys.executable,
+        ]
+        + sys.argv,
+        env=env,
+        start_new_session=True,
+    )
+    logger.info(f"🚀 EOS restarted, PID {new_process.pid}")
+
+    # Gracefully shut down this process.
+    asyncio.create_task(server_shutdown_task())
+
+    # Will be executed because shutdown is delegated to async coroutine
+    return {
+        "message": "Restarting EOS...",
+        "pid": new_process.pid,
+    }
+
+
+@app.post("/v1/admin/server/shutdown", tags=["admin"])
+async def fastapi_admin_server_shutdown_post() -> dict:
+    """Shutdown the server."""
+    logger.info("🔄 Stopping EOS...")
+
+    # Gracefully shut down this process.
+    asyncio.create_task(server_shutdown_task())
+
+    # Will be executed because shutdown is delegated to async coroutine
+    return {
+        "message": "Stopping EOS...",
+        "pid": psutil.Process().pid,
+    }
+
+
+@app.get("/v1/health")
+def fastapi_health_get():  # type: ignore
+    """Health check endpoint to verify that the EOS server is alive."""
+    return JSONResponse(
+        {
+            "status": "alive",
+            "pid": psutil.Process().pid,
+        }
+    )
+
+
+@app.post("/v1/config/reset", tags=["config"])
+def fastapi_config_reset_post() -> ConfigEOS:
+    """Reset the configuration.
 
     Returns:
         configuration (ConfigEOS): The current configuration after update.
@@ -251,7 +533,7 @@ def fastapi_config_update_post() -> ConfigEOS:
     except Exception as e:
         raise HTTPException(
             status_code=404,
-            detail=f"Cannot update configuration from file '{config_eos.config_file_path}': {e}",
+            detail=f"Cannot reset configuration: {e}",
         )
     return config_eos
 
@@ -543,7 +825,7 @@ def fastapi_prediction_list_get(
     ] = None,
     interval: Annotated[
         Optional[str],
-        Query(description="Time duration for each interval."),
+        Query(description="Time duration for each interval. Defaults to 1 hour."),
     ] = None,
 ) -> List[Any]:
     """Get prediction for given key within given date range as value list.
@@ -580,8 +862,40 @@ def fastapi_prediction_list_get(
     return prediction_list
 
 
+@app.put("/v1/prediction/import/{provider_id}", tags=["prediction"])
+def fastapi_prediction_import_provider(
+    provider_id: str = FastapiPath(..., description="Provider ID."),
+    data: Optional[Union[PydanticDateTimeDataFrame, PydanticDateTimeData, dict]] = None,
+    force_enable: Optional[bool] = None,
+) -> Response:
+    """Import prediction for given provider ID.
+
+    Args:
+        provider_id: ID of provider to update.
+        data: Prediction data.
+        force_enable: Update data even if provider is disabled.
+            Defaults to False.
+    """
+    try:
+        provider = prediction_eos.provider_by_id(provider_id)
+    except ValueError:
+        raise HTTPException(status_code=404, detail=f"Provider '{provider_id}' not found.")
+    if not provider.enabled() and not force_enable:
+        raise HTTPException(status_code=404, detail=f"Provider '{provider_id}' not enabled.")
+    try:
+        provider.import_from_json(json_str=json.dumps(data))
+        provider.update_datetime = to_datetime(in_timezone=config_eos.general.timezone)
+    except Exception as e:
+        raise HTTPException(
+            status_code=400, detail=f"Error on import for provider '{provider_id}': {e}"
+        )
+    return Response()
+
+
 @app.post("/v1/prediction/update", tags=["prediction"])
-def fastapi_prediction_update(force_update: bool = False, force_enable: bool = False) -> Response:
+def fastapi_prediction_update(
+    force_update: Optional[bool] = False, force_enable: Optional[bool] = False
+) -> Response:
     """Update predictions for all providers.
 
     Args:
@@ -593,8 +907,7 @@ def fastapi_prediction_update(force_update: bool = False, force_enable: bool = F
     try:
         prediction_eos.update_data(force_update=force_update, force_enable=force_enable)
     except Exception as e:
-        raise e
-        # raise HTTPException(status_code=400, detail=f"Error on update of provider: {e}")
+        raise HTTPException(status_code=400, detail=f"Error on prediction update: {e}")
     return Response()
 
 
@@ -912,34 +1225,35 @@ def site_map() -> RedirectResponse:
 
 # Keep the proxy last to handle all requests that are not taken by the Rest API.
 
-if config_eos.server.startup_eosdash:
 
-    @app.delete("/{path:path}", include_in_schema=False)
-    async def proxy_delete(request: Request, path: str) -> Response:
-        return await proxy(request, path)
+@app.delete("/{path:path}", include_in_schema=False)
+async def proxy_delete(request: Request, path: str) -> Response:
+    return await proxy(request, path)
 
-    @app.get("/{path:path}", include_in_schema=False)
-    async def proxy_get(request: Request, path: str) -> Response:
-        return await proxy(request, path)
 
-    @app.post("/{path:path}", include_in_schema=False)
-    async def proxy_post(request: Request, path: str) -> Response:
-        return await proxy(request, path)
+@app.get("/{path:path}", include_in_schema=False)
+async def proxy_get(request: Request, path: str) -> Response:
+    return await proxy(request, path)
 
-    @app.put("/{path:path}", include_in_schema=False)
-    async def proxy_put(request: Request, path: str) -> Response:
-        return await proxy(request, path)
-else:
 
-    @app.get("/", include_in_schema=False)
-    def root() -> RedirectResponse:
-        return RedirectResponse(url="/docs")
+@app.post("/{path:path}", include_in_schema=False)
+async def proxy_post(request: Request, path: str) -> Response:
+    return await proxy(request, path)
+
+
+@app.put("/{path:path}", include_in_schema=False)
+async def proxy_put(request: Request, path: str) -> Response:
+    return await proxy(request, path)
 
 
 async def proxy(request: Request, path: str) -> Union[Response | RedirectResponse | HTMLResponse]:
-    if config_eos.server.eosdash_host and config_eos.server.eosdash_port:
+    # Make hostname Windows friendly
+    host = str(config_eos.server.eosdash_host)
+    if host == "0.0.0.0" and os.name == "nt":
+        host = "localhost"
+    if host and config_eos.server.eosdash_port:
         # Proxy to EOSdash server
-        url = f"http://{config_eos.server.eosdash_host}:{config_eos.server.eosdash_port}/{path}"
+        url = f"http://{host}:{config_eos.server.eosdash_port}/{path}"
         headers = dict(request.headers)
 
         data = await request.body()
@@ -1004,6 +1318,29 @@ def run_eos(host: str, port: int, log_level: str, access_log: bool, reload: bool
     # Make hostname Windows friendly
     if host == "0.0.0.0" and os.name == "nt":
         host = "localhost"
+
+    # Wait for EOS port to be free - e.g. in case of restart
+    timeout = 120  # Maximum 120 seconds to wait
+    process_info: list[dict] = []
+    for retries in range(int(timeout / 10)):
+        process_info = []
+        pids: list[int] = []
+        for conn in psutil.net_connections(kind="inet"):
+            if conn.laddr.port == port:
+                if conn.pid not in pids:
+                    # Get fresh process info
+                    process = psutil.Process(conn.pid)
+                    pids.append(conn.pid)
+                    process_info.append(process.as_dict(attrs=["pid", "cmdline"]))
+        if len(process_info) == 0:
+            break
+        logger.info(f"EOS waiting for port `{port}` ...")
+        time.sleep(10)
+    if len(process_info) > 0:
+        logger.warning(f"EOS port `{port}` in use.")
+        for info in process_info:
+            logger.warning(f"PID: `{info["pid"]}`, CMD: `{info["cmdline"]}`")
+
     try:
         uvicorn.run(
             "akkudoktoreos.server.eos:app",
@@ -1071,8 +1408,11 @@ def main() -> None:
 
     args = parser.parse_args()
 
+    host = args.host if args.host else get_default_host()
+    port = args.port if args.port else 8503
+
     try:
-        run_eos(args.host, args.port, args.log_level, args.access_log, args.reload)
+        run_eos(host, port, args.log_level, args.access_log, args.reload)
     except:
         sys.exit(1)
 

src/akkudoktoreos/server/eosdash.py

@@ -1,11 +1,14 @@
 import argparse
 import os
 import sys
+import time
 from functools import reduce
 from typing import Any, Union
 
+import psutil
 import uvicorn
 from fasthtml.common import H1, Table, Td, Th, Thead, Titled, Tr, fast_app
+from fasthtml.starlette import JSONResponse
 from pydantic.fields import ComputedFieldInfo, FieldInfo
 from pydantic_core import PydanticUndefined
 
@@ -121,6 +124,17 @@ def get():  # type: ignore
     return Titled("EOS Dashboard", H1("Configuration"), config_table())
 
 
+@app.get("/eosdash/health")
+def get_eosdash_health():  # type: ignore
+    """Health check endpoint to verify that the EOSdash server is alive."""
+    return JSONResponse(
+        {
+            "status": "alive",
+            "pid": psutil.Process().pid,
+        }
+    )
+
+
 def run_eosdash(host: str, port: int, log_level: str, access_log: bool, reload: bool) -> None:
     """Run the EOSdash server with the specified configurations.
 
@@ -131,20 +145,54 @@ def run_eosdash(host: str, port: int, log_level: str, access_log: bool, reload:
     server to the specified host and port, an error message is logged and the
     application exits.
 
-    Parameters:
-    host (str): The hostname to bind the server to.
-    port (int): The port number to bind the server to.
-    log_level (str): The log level for the server. Options include "critical", "error",
-                     "warning", "info", "debug", and "trace".
-    access_log (bool): Whether to enable or disable the access log. Set to True to enable.
-    reload (bool): Whether to enable or disable auto-reload. Set to True for development.
+    Args:
+        host (str): The hostname to bind the server to.
+        port (int): The port number to bind the server to.
+        log_level (str): The log level for the server. Options include "critical", "error",
+                        "warning", "info", "debug", and "trace".
+        access_log (bool): Whether to enable or disable the access log. Set to True to enable.
+        reload (bool): Whether to enable or disable auto-reload. Set to True for development.
 
     Returns:
-    None
+        None
     """
     # Make hostname Windows friendly
     if host == "0.0.0.0" and os.name == "nt":
         host = "localhost"
+
+    # Wait for EOSdash port to be free - e.g. in case of restart
+    timeout = 120  # Maximum 120 seconds to wait
+    process_info: list[dict] = []
+    for retries in range(int(timeout / 3)):
+        process_info = []
+        pids: list[int] = []
+        for conn in psutil.net_connections(kind="inet"):
+            if conn.laddr.port == port:
+                if conn.pid not in pids:
+                    # Get fresh process info
+                    process = psutil.Process(conn.pid)
+                    pids.append(conn.pid)
+                    process_info.append(process.as_dict(attrs=["pid", "cmdline"]))
+        if len(process_info) == 0:
+            break
+        logger.info(f"EOSdash waiting for port `{port}` ...")
+        time.sleep(3)
+    if len(process_info) > 0:
+        logger.warning(f"EOSdash port `{port}` in use.")
+        for info in process_info:
+            logger.warning(f"PID: `{info["pid"]}`, CMD: `{info["cmdline"]}`")
+
+    # Setup config from args
+    if args:
+        if args.eos_host:
+            config_eos.server.host = args.eos_host
+        if args.eos_port:
+            config_eos.server.port = args.eos_port
+        if args.host:
+            config_eos.server.eosdash_host = args.host
+        if args.port:
+            config_eos.server.eosdash_port = args.port
+
     try:
         uvicorn.run(
             "akkudoktoreos.server.eosdash:app",
@@ -197,13 +245,13 @@ def main() -> None:
         "--eos-host",
         type=str,
         default=str(config_eos.server.host),
-        help="Host for the EOS server (default: value from config)",
+        help="Host of the EOS server (default: value from config)",
     )
     parser.add_argument(
         "--eos-port",
         type=int,
         default=config_eos.server.port,
-        help="Port for the EOS server (default: value from config)",
+        help="Port of the EOS server (default: value from config)",
     )
 
     # Optional arguments for log_level, access_log, and reload
@@ -230,7 +278,9 @@ def main() -> None:
 
     try:
         run_eosdash(args.host, args.port, args.log_level, args.access_log, args.reload)
-    except:
+    except Exception as ex:
+        error_msg = f"Failed to run EOSdash: {ex}"
+        logger.error(error_msg)
         sys.exit(1)
 
 

src/akkudoktoreos/server/rest/tasks.py

@@ -0,0 +1,92 @@
+"""Task handling taken from fastapi-utils/fastapi_utils/tasks.py."""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from functools import wraps
+from typing import Any, Callable, Coroutine, Union
+
+from starlette.concurrency import run_in_threadpool
+
+NoArgsNoReturnFuncT = Callable[[], None]
+NoArgsNoReturnAsyncFuncT = Callable[[], Coroutine[Any, Any, None]]
+ExcArgNoReturnFuncT = Callable[[Exception], None]
+ExcArgNoReturnAsyncFuncT = Callable[[Exception], Coroutine[Any, Any, None]]
+NoArgsNoReturnAnyFuncT = Union[NoArgsNoReturnFuncT, NoArgsNoReturnAsyncFuncT]
+ExcArgNoReturnAnyFuncT = Union[ExcArgNoReturnFuncT, ExcArgNoReturnAsyncFuncT]
+NoArgsNoReturnDecorator = Callable[[NoArgsNoReturnAnyFuncT], NoArgsNoReturnAsyncFuncT]
+
+
+async def _handle_func(func: NoArgsNoReturnAnyFuncT) -> None:
+    if asyncio.iscoroutinefunction(func):
+        await func()
+    else:
+        await run_in_threadpool(func)
+
+
+async def _handle_exc(exc: Exception, on_exception: ExcArgNoReturnAnyFuncT | None) -> None:
+    if on_exception:
+        if asyncio.iscoroutinefunction(on_exception):
+            await on_exception(exc)
+        else:
+            await run_in_threadpool(on_exception, exc)
+
+
+def repeat_every(
+    *,
+    seconds: float,
+    wait_first: float | None = None,
+    logger: logging.Logger | None = None,
+    raise_exceptions: bool = False,
+    max_repetitions: int | None = None,
+    on_complete: NoArgsNoReturnAnyFuncT | None = None,
+    on_exception: ExcArgNoReturnAnyFuncT | None = None,
+) -> NoArgsNoReturnDecorator:
+    """A decorator that modifies a function so it is periodically re-executed after its first call.
+
+    The function it decorates should accept no arguments and return nothing. If necessary, this can be accomplished
+    by using `functools.partial` or otherwise wrapping the target function prior to decoration.
+
+    Parameters
+    ----------
+    seconds: float
+        The number of seconds to wait between repeated calls
+    wait_first: float (default None)
+        If not None, the function will wait for the given duration before the first call
+    max_repetitions: Optional[int] (default None)
+        The maximum number of times to call the repeated function. If `None`, the function is repeated forever.
+    on_complete: Optional[Callable[[], None]] (default None)
+        A function to call after the final repetition of the decorated function.
+    on_exception: Optional[Callable[[Exception], None]] (default None)
+        A function to call when an exception is raised by the decorated function.
+    """
+
+    def decorator(func: NoArgsNoReturnAnyFuncT) -> NoArgsNoReturnAsyncFuncT:
+        """Converts the decorated function into a repeated, periodically-called version."""
+
+        @wraps(func)
+        async def wrapped() -> None:
+            async def loop() -> None:
+                if wait_first is not None:
+                    await asyncio.sleep(wait_first)
+
+                repetitions = 0
+                while max_repetitions is None or repetitions < max_repetitions:
+                    try:
+                        await _handle_func(func)
+
+                    except Exception as exc:
+                        await _handle_exc(exc, on_exception)
+
+                    repetitions += 1
+                    await asyncio.sleep(seconds)
+
+                if on_complete:
+                    await _handle_func(on_complete)
+
+            asyncio.ensure_future(loop())
+
+        return wrapped
+
+    return decorator

src/akkudoktoreos/server/server.py

@@ -1,5 +1,6 @@
 """Server Module."""
 
+import os
 from typing import Optional
 
 from pydantic import Field, IPvAnyAddress, field_validator
@@ -10,6 +11,12 @@ from akkudoktoreos.core.logging import get_logger
 logger = get_logger(__name__)
 
 
+def get_default_host() -> str:
+    if os.name == "nt":
+        return "127.0.0.1"
+    return "0.0.0.0"
+
+
 class ServerCommonSettings(SettingsBaseModel):
     """Server Configuration.
 
@@ -17,14 +24,16 @@ class ServerCommonSettings(SettingsBaseModel):
         To be added
     """
 
-    host: Optional[IPvAnyAddress] = Field(default="0.0.0.0", description="EOS server IP address.")
+    host: Optional[IPvAnyAddress] = Field(
+        default=get_default_host(), description="EOS server IP address."
+    )
     port: Optional[int] = Field(default=8503, description="EOS server IP port number.")
     verbose: Optional[bool] = Field(default=False, description="Enable debug output")
     startup_eosdash: Optional[bool] = Field(
         default=True, description="EOS server to start EOSdash server."
     )
     eosdash_host: Optional[IPvAnyAddress] = Field(
-        default="0.0.0.0", description="EOSdash server IP address."
+        default=get_default_host(), description="EOSdash server IP address."
     )
     eosdash_port: Optional[int] = Field(default=8504, description="EOSdash server IP port number.")
 

src/akkudoktoreos/utils/visualize.py

@@ -46,7 +46,7 @@ class VisualizationReport(ConfigMixin):
         """Add a chart function to the current group and save it as a PNG and SVG."""
         self.current_group.append(chart_func)
         if self.create_img and title:
-            server_output_dir = self.config.general.data_cache_path
+            server_output_dir = self.config.cache.path()
             server_output_dir.mkdir(parents=True, exist_ok=True)
             fig, ax = plt.subplots()
             chart_func()

tests/conftest.py

@@ -1,18 +1,26 @@
+import json
 import logging
 import os
+import signal
 import subprocess
 import sys
 import tempfile
+import time
+from contextlib import contextmanager
+from http import HTTPStatus
 from pathlib import Path
-from typing import Optional
+from typing import Generator, Optional, Union
 from unittest.mock import PropertyMock, patch
 
 import pendulum
+import psutil
 import pytest
-from xprocess import ProcessStarter
+import requests
+from xprocess import ProcessStarter, XProcess
 
 from akkudoktoreos.config.config import ConfigEOS, get_config
 from akkudoktoreos.core.logging import get_logger
+from akkudoktoreos.server.server import get_default_host
 
 logger = get_logger(__name__)
 
@@ -48,6 +56,12 @@ def pytest_addoption(parser):
         default=False,
         help="Verify that user config file is non-existent (will also fail if user config file exists before test run).",
     )
+    parser.addoption(
+        "--system-test",
+        action="store_true",
+        default=False,
+        help="System test mode. Tests may access real resources, like prediction providers!",
+    )
 
 
 @pytest.fixture
@@ -64,6 +78,18 @@ def config_mixin(config_eos):
         yield config_mixin_patch
 
 
+@pytest.fixture
+def is_system_test(request):
+    yield bool(request.config.getoption("--system-test"))
+
+
+@pytest.fixture
+def prediction_eos():
+    from akkudoktoreos.prediction.prediction import get_prediction
+
+    return get_prediction()
+
+
 @pytest.fixture
 def devices_eos(config_mixin):
     from akkudoktoreos.devices.devices import get_devices
@@ -87,13 +113,33 @@ def devices_mixin(devices_eos):
 # Before activating, make sure that no user config file exists (e.g. ~/.config/net.akkudoktoreos.eos/EOS.config.json)
 @pytest.fixture(autouse=True)
 def cfg_non_existent(request):
-    yield
-    if bool(request.config.getoption("--check-config-side-effect")):
-        from platformdirs import user_config_dir
+    if not bool(request.config.getoption("--check-config-side-effect")):
+        yield
+        return
 
-        user_dir = user_config_dir(ConfigEOS.APP_NAME)
-        assert not Path(user_dir).joinpath(ConfigEOS.CONFIG_FILE_NAME).exists()
-        assert not Path.cwd().joinpath(ConfigEOS.CONFIG_FILE_NAME).exists()
+    # Before test
+    from platformdirs import user_config_dir
+
+    user_dir = user_config_dir(ConfigEOS.APP_NAME)
+    user_config_file = Path(user_dir).joinpath(ConfigEOS.CONFIG_FILE_NAME)
+    cwd_config_file = Path.cwd().joinpath(ConfigEOS.CONFIG_FILE_NAME)
+    assert (
+        not user_config_file.exists()
+    ), f"Config file {user_config_file} exists, please delete before test!"
+    assert (
+        not cwd_config_file.exists()
+    ), f"Config file {cwd_config_file} exists, please delete before test!"
+
+    # Yield to test
+    yield
+
+    # After test
+    assert (
+        not user_config_file.exists()
+    ), f"Config file {user_config_file} created, please check test!"
+    assert (
+        not cwd_config_file.exists()
+    ), f"Config file {cwd_config_file} created, please check test!"
 
 
 @pytest.fixture(autouse=True)
@@ -149,52 +195,252 @@ def config_eos(
     assert config_file.exists()
     assert not config_file_cwd.exists()
     assert config_default_dirs[-1] / "data" == config_eos.general.data_folder_path
-    assert config_default_dirs[-1] / "data/cache" == config_eos.general.data_cache_path
+    assert config_default_dirs[-1] / "data/cache" == config_eos.cache.path()
     assert config_default_dirs[-1] / "data/output" == config_eos.general.data_output_path
     return config_eos
 
 
 @pytest.fixture
-def config_default_dirs():
+def config_default_dirs(tmpdir):
     """Fixture that provides a list of directories to be used as config dir."""
-    with tempfile.TemporaryDirectory() as tmp_user_home_dir:
-        # Default config directory from platform user config directory
-        config_default_dir_user = Path(tmp_user_home_dir) / "config"
+    tmp_user_home_dir = Path(tmpdir)
 
-        # Default config directory from current working directory
-        config_default_dir_cwd = Path(tmp_user_home_dir) / "cwd"
-        config_default_dir_cwd.mkdir()
+    # Default config directory from platform user config directory
+    config_default_dir_user = tmp_user_home_dir / "config"
 
-        # Default config directory from default config file
-        config_default_dir_default = Path(__file__).parent.parent.joinpath("src/akkudoktoreos/data")
+    # Default config directory from current working directory
+    config_default_dir_cwd = tmp_user_home_dir / "cwd"
+    config_default_dir_cwd.mkdir()
 
-        # Default data directory from platform user data directory
-        data_default_dir_user = Path(tmp_user_home_dir)
-        yield (
-            config_default_dir_user,
-            config_default_dir_cwd,
-            config_default_dir_default,
-            data_default_dir_user,
-        )
+    # Default config directory from default config file
+    config_default_dir_default = Path(__file__).parent.parent.joinpath("src/akkudoktoreos/data")
+
+    # Default data directory from platform user data directory
+    data_default_dir_user = tmp_user_home_dir
+
+    return (
+        config_default_dir_user,
+        config_default_dir_cwd,
+        config_default_dir_default,
+        data_default_dir_user,
+    )
+
+
+@contextmanager
+def server_base(xprocess: XProcess) -> Generator[dict[str, Union[str, int]], None, None]:
+    """Fixture to start the server with temporary EOS_DIR and default config.
+
+    Args:
+        xprocess (XProcess): The pytest-xprocess fixture to manage the server process.
+
+    Yields:
+        dict[str, str]: A dictionary containing:
+            - "server" (str): URL of the server.
+            - "eos_dir" (str): Path to the temporary EOS_DIR.
+    """
+    host = get_default_host()
+    port = 8503
+    eosdash_port = 8504
+
+    # Port of server may be still blocked by a server usage despite the other server already
+    # shut down. CLOSE_WAIT, TIME_WAIT may typically take up to 120 seconds.
+    server_timeout = 120
+
+    server = f"http://{host}:{port}"
+    eosdash_server = f"http://{host}:{eosdash_port}"
+    eos_tmp_dir = tempfile.TemporaryDirectory()
+    eos_dir = str(eos_tmp_dir.name)
+
+    class Starter(ProcessStarter):
+        # assure server to be installed
+        try:
+            project_dir = Path(__file__).parent.parent
+            subprocess.run(
+                [sys.executable, "-c", "import", "akkudoktoreos.server.eos"],
+                check=True,
+                env=os.environ,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                cwd=project_dir,
+            )
+        except subprocess.CalledProcessError:
+            subprocess.run(
+                [sys.executable, "-m", "pip", "install", "-e", str(project_dir)],
+                env=os.environ,
+                check=True,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                cwd=project_dir,
+            )
+
+        # Set environment for server run
+        env = os.environ.copy()
+        env["EOS_DIR"] = eos_dir
+        env["EOS_CONFIG_DIR"] = eos_dir
+
+        # command to start server process
+        args = [
+            sys.executable,
+            "-m",
+            "akkudoktoreos.server.eos",
+            "--host",
+            host,
+            "--port",
+            str(port),
+        ]
+
+        # Will wait for 'server_timeout' seconds before timing out
+        timeout = server_timeout
+
+        # xprocess will now attempt to clean up upon interruptions
+        terminate_on_interrupt = True
+
+        # checks if our server is ready
+        def startup_check(self):
+            try:
+                result = requests.get(f"{server}/v1/health", timeout=2)
+                if result.status_code == 200:
+                    return True
+            except:
+                pass
+            return False
+
+    def cleanup_eos_eosdash():
+        # Cleanup any EOS process left.
+        if os.name == "nt":
+            # Windows does not provide SIGKILL
+            sigkill = signal.SIGTERM
+        else:
+            sigkill = signal.SIGKILL
+        # - Use pid on EOS health endpoint
+        try:
+            result = requests.get(f"{server}/v1/health", timeout=2)
+            if result.status_code == HTTPStatus.OK:
+                pid = result.json()["pid"]
+                os.kill(pid, sigkill)
+                time.sleep(1)
+                result = requests.get(f"{server}/v1/health", timeout=2)
+                assert result.status_code != HTTPStatus.OK
+        except:
+            pass
+        # - Use pids from processes on EOS port
+        for retries in range(int(server_timeout / 3)):
+            pids: list[int] = []
+            for conn in psutil.net_connections(kind="inet"):
+                if conn.laddr.port == port:
+                    if conn.pid not in pids:
+                        # Get fresh process info
+                        try:
+                            process = psutil.Process(conn.pid)
+                            process_info = process.as_dict(attrs=["pid", "cmdline"])
+                            if "akkudoktoreos.server.eos" in process_info["cmdline"]:
+                                pids.append(conn.pid)
+                        except:
+                            # PID may already be dead
+                            pass
+                for pid in pids:
+                    os.kill(pid, sigkill)
+            if len(pids) == 0:
+                break
+            time.sleep(3)
+        assert len(pids) == 0
+        # Cleanup any EOSdash processes left.
+        # - Use pid on EOSdash health endpoint
+        try:
+            result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+            if result.status_code == HTTPStatus.OK:
+                pid = result.json()["pid"]
+                os.kill(pid, sigkill)
+                time.sleep(1)
+                result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+                assert result.status_code != HTTPStatus.OK
+        except:
+            pass
+        # - Use pids from processes on EOSdash port
+        for retries in range(int(server_timeout / 3)):
+            pids = []
+            for conn in psutil.net_connections(kind="inet"):
+                if conn.laddr.port == eosdash_port:
+                    if conn.pid not in pids:
+                        # Get fresh process info
+                        try:
+                            process = psutil.Process(conn.pid)
+                            process_info = process.as_dict(attrs=["pid", "cmdline"])
+                            if "akkudoktoreos.server.eosdash" in process_info["cmdline"]:
+                                pids.append(conn.pid)
+                        except:
+                            # PID may already be dead
+                            pass
+            for pid in pids:
+                os.kill(pid, sigkill)
+            if len(pids) == 0:
+                break
+            time.sleep(3)
+        assert len(pids) == 0
+
+    # Kill all running eos and eosdash process - just to be sure
+    cleanup_eos_eosdash()
+
+    # Ensure there is an empty config file in the temporary EOS directory
+    config_file_path = Path(eos_dir).joinpath(ConfigEOS.CONFIG_FILE_NAME)
+    with config_file_path.open(mode="w", encoding="utf-8", newline="\n") as fd:
+        json.dump({}, fd)
+
+    # ensure process is running and return its logfile
+    pid, logfile = xprocess.ensure("eos", Starter)
+    logger.info(f"Started EOS ({pid}). This may take very long (up to {server_timeout} seconds).")
+    logger.info(f"View xprocess logfile at: {logfile}")
+
+    yield {
+        "server": server,
+        "eosdash_server": eosdash_server,
+        "eos_dir": eos_dir,
+        "timeout": server_timeout,
+    }
+
+    # clean up whole process tree afterwards
+    xprocess.getinfo("eos").terminate()
+
+    # Cleanup any EOS process left.
+    cleanup_eos_eosdash()
+
+    # Remove temporary EOS_DIR
+    eos_tmp_dir.cleanup()
+
+
+@pytest.fixture(scope="class")
+def server_setup_for_class(xprocess) -> Generator[dict[str, Union[str, int]], None, None]:
+    """A fixture to start the server for a test class."""
+    with server_base(xprocess) as result:
+        yield result
+
+
+@pytest.fixture(scope="function")
+def server_setup_for_function(xprocess) -> Generator[dict[str, Union[str, int]], None, None]:
+    """A fixture to start the server for a test function."""
+    with server_base(xprocess) as result:
+        yield result
 
 
 @pytest.fixture
-def server(xprocess, config_eos, config_default_dirs):
+def server(xprocess, config_eos, config_default_dirs) -> Generator[str, None, None]:
     """Fixture to start the server.
 
     Provides URL of the server.
     """
+    # create url/port info to the server
+    url = "http://0.0.0.0:8503"
 
     class Starter(ProcessStarter):
         # Set environment before any subprocess run, to keep custom config dir
         env = os.environ.copy()
         env["EOS_DIR"] = str(config_default_dirs[-1])
-        project_dir = config_eos.package_root_path
+        project_dir = config_eos.package_root_path.parent.parent
 
         # assure server to be installed
         try:
             subprocess.run(
-                [sys.executable, "-c", "import akkudoktoreos.server.eos"],
+                [sys.executable, "-c", "import", "akkudoktoreos.server.eos"],
                 check=True,
                 env=env,
                 stdout=subprocess.PIPE,
@@ -203,7 +449,7 @@ def server(xprocess, config_eos, config_default_dirs):
             )
         except subprocess.CalledProcessError:
             subprocess.run(
-                [sys.executable, "-m", "pip", "install", "-e", project_dir],
+                [sys.executable, "-m", "pip", "install", "-e", str(project_dir)],
                 check=True,
                 stdout=subprocess.PIPE,
                 stderr=subprocess.PIPE,
@@ -212,24 +458,26 @@ def server(xprocess, config_eos, config_default_dirs):
         # command to start server process
         args = [sys.executable, "-m", "akkudoktoreos.server.eos"]
 
-        # startup pattern
-        pattern = "Application startup complete."
-        # search this number of lines for the startup pattern, if not found
-        # a RuntimeError will be raised informing the user
-        max_read_lines = 30
-
-        # will wait for 30 seconds before timing out
-        timeout = 30
+        # will wait for xx seconds before timing out
+        timeout = 10
 
         # xprocess will now attempt to clean up upon interruptions
         terminate_on_interrupt = True
 
+        # checks if our server is ready
+        def startup_check(self):
+            try:
+                result = requests.get(f"{url}/v1/health")
+                if result.status_code == 200:
+                    return True
+            except:
+                pass
+            return False
+
     # ensure process is running and return its logfile
     pid, logfile = xprocess.ensure("eos", Starter)
     print(f"View xprocess logfile at: {logfile}")
 
-    # create url/port info to the server
-    url = "http://127.0.0.1:8503"
     yield url
 
     # clean up whole process tree afterwards

tests/test_cache.py

@@ -0,0 +1,694 @@
+import io
+import json
+import pickle
+import tempfile
+from datetime import date, datetime, timedelta
+from pathlib import Path
+from time import sleep
+from unittest.mock import MagicMock, patch
+
+import cachebox
+import pytest
+
+from akkudoktoreos.core.cache import (
+    CacheFileRecord,
+    CacheFileStore,
+    CacheUntilUpdateStore,
+    cache_in_file,
+    cache_until_update,
+    cachemethod_until_update,
+)
+from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration
+
+# ---------------------------------
+# In-Memory Caching Functionality
+# ---------------------------------
+
+
+# Fixtures for testing
+@pytest.fixture
+def cache_until_update_store():
+    """Ensures CacheUntilUpdateStore is reset between tests."""
+    cache = CacheUntilUpdateStore()
+    CacheUntilUpdateStore().clear()
+    assert len(cache) == 0
+    return cache
+
+
+class TestCacheUntilUpdateStore:
+    def test_cache_initialization(self, cache_until_update_store):
+        """Test that CacheUntilUpdateStore initializes with the correct properties."""
+        cache = CacheUntilUpdateStore()
+        assert isinstance(cache.cache, cachebox.LRUCache)
+        assert cache.maxsize == 100
+        assert len(cache) == 0
+
+    def test_singleton_behavior(self, cache_until_update_store):
+        """Test that CacheUntilUpdateStore is a singleton."""
+        cache1 = CacheUntilUpdateStore()
+        cache2 = CacheUntilUpdateStore()
+        assert cache1 is cache2
+
+    def test_cache_storage(self, cache_until_update_store):
+        """Test that items can be added and retrieved from the cache."""
+        cache = CacheUntilUpdateStore()
+        cache["key1"] = "value1"
+        assert cache["key1"] == "value1"
+        assert len(cache) == 1
+
+    def test_cache_getattr_invalid_method(self, cache_until_update_store):
+        """Test that accessing an invalid method raises an AttributeError."""
+        with pytest.raises(AttributeError):
+            CacheUntilUpdateStore().non_existent_method()  # This should raise AttributeError
+
+
+class TestCacheUntilUpdateDecorators:
+    def test_cachemethod_until_update(self, cache_until_update_store):
+        """Test that cachemethod_until_update caches method results."""
+
+        class MyClass:
+            @cachemethod_until_update
+            def compute(self, value: int) -> int:
+                return value * 2
+
+        obj = MyClass()
+
+        # Call method and assert caching
+        assert CacheUntilUpdateStore.miss_count == 0
+        assert CacheUntilUpdateStore.hit_count == 0
+        result1 = obj.compute(5)
+        assert CacheUntilUpdateStore.miss_count == 1
+        assert CacheUntilUpdateStore.hit_count == 0
+        result2 = obj.compute(5)
+        assert CacheUntilUpdateStore.miss_count == 1
+        assert CacheUntilUpdateStore.hit_count == 1
+        assert result1 == result2
+
+    def test_cache_until_update(self, cache_until_update_store):
+        """Test that cache_until_update caches function results."""
+
+        @cache_until_update
+        def compute(value: int) -> int:
+            return value * 3
+
+        # Call function and assert caching
+        result1 = compute(4)
+        assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS
+        result2 = compute(4)
+        assert CacheUntilUpdateStore.last_event == cachebox.EVENT_HIT
+        assert result1 == result2
+
+    def test_cache_with_different_arguments(self, cache_until_update_store):
+        """Test that caching works for different arguments."""
+
+        class MyClass:
+            @cachemethod_until_update
+            def compute(self, value: int) -> int:
+                return value * 2
+
+        obj = MyClass()
+
+        assert CacheUntilUpdateStore.miss_count == 0
+        result1 = obj.compute(3)
+        assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS
+        assert CacheUntilUpdateStore.miss_count == 1
+        result2 = obj.compute(5)
+        assert CacheUntilUpdateStore.last_event == cachebox.EVENT_MISS
+        assert CacheUntilUpdateStore.miss_count == 2
+
+        assert result1 == 6
+        assert result2 == 10
+
+    def test_cache_clearing(self, cache_until_update_store):
+        """Test that cache is cleared between EMS update cycles."""
+
+        class MyClass:
+            @cachemethod_until_update
+            def compute(self, value: int) -> int:
+                return value * 2
+
+        obj = MyClass()
+        obj.compute(5)
+
+        # Clear cache
+        CacheUntilUpdateStore().clear()
+
+        with pytest.raises(KeyError):
+            _ = CacheUntilUpdateStore()["<invalid>"]
+
+    def test_decorator_works_for_standalone_function(self, cache_until_update_store):
+        """Test that cache_until_update works with standalone functions."""
+
+        @cache_until_update
+        def add(a: int, b: int) -> int:
+            return a + b
+
+        assert CacheUntilUpdateStore.miss_count == 0
+        assert CacheUntilUpdateStore.hit_count == 0
+        result1 = add(1, 2)
+        assert CacheUntilUpdateStore.miss_count == 1
+        assert CacheUntilUpdateStore.hit_count == 0
+        result2 = add(1, 2)
+        assert CacheUntilUpdateStore.miss_count == 1
+        assert CacheUntilUpdateStore.hit_count == 1
+
+        assert result1 == result2
+
+
+# -----------------------------
+# CacheFileStore
+# -----------------------------
+
+
+@pytest.fixture
+def temp_store_file():
+    with tempfile.NamedTemporaryFile(delete=False) as temp_file:
+        yield Path(temp_file.file.name)
+    # temp_file.unlink()
+
+
+@pytest.fixture
+def cache_file_store(temp_store_file):
+    """A pytest fixture that creates a new CacheFileStore instance for testing."""
+    cache = CacheFileStore()
+    cache._store_file = temp_store_file
+    cache.clear(clear_all=True)
+    assert len(cache._store) == 0
+    return cache
+
+
+class TestCacheFileStore:
+    def test_generate_cache_file_key(self, cache_file_store):
+        """Test cache file key generation based on URL and date."""
+        key = "http://example.com"
+
+        # Provide until date - assure until_dt is used.
+        until_dt = to_datetime("2024-10-01")
+        cache_file_key, cache_file_until_dt, ttl_duration = (
+            cache_file_store._generate_cache_file_key(key=key, until_datetime=until_dt)
+        )
+        assert cache_file_key is not None
+        assert compare_datetimes(cache_file_until_dt, until_dt).equal
+
+        # Provide until date again - assure same key is generated.
+        cache_file_key1, cache_file_until_dt1, ttl_duration1 = (
+            cache_file_store._generate_cache_file_key(key=key, until_datetime=until_dt)
+        )
+        assert cache_file_key1 == cache_file_key
+        assert compare_datetimes(cache_file_until_dt1, until_dt).equal
+
+        # Provide no until date - assure today EOD is used.
+        no_until_dt = to_datetime().end_of("day")
+        cache_file_key, cache_file_until_dt, ttl_duration = (
+            cache_file_store._generate_cache_file_key(key)
+        )
+        assert cache_file_key is not None
+        assert compare_datetimes(cache_file_until_dt, no_until_dt).equal
+
+        # Provide with_ttl - assure until_dt is used.
+        until_dt = to_datetime().add(hours=1)
+        cache_file_key, cache_file_until_dt, ttl_duration = (
+            cache_file_store._generate_cache_file_key(key, with_ttl="1 hour")
+        )
+        assert cache_file_key is not None
+        assert compare_datetimes(cache_file_until_dt, until_dt).approximately_equal
+        assert ttl_duration == to_duration("1 hour")
+
+        # Provide with_ttl again - assure same key is generated.
+        until_dt = to_datetime().add(hours=1)
+        cache_file_key1, cache_file_until_dt1, ttl_duration1 = (
+            cache_file_store._generate_cache_file_key(key=key, with_ttl="1 hour")
+        )
+        assert cache_file_key1 == cache_file_key
+        assert compare_datetimes(cache_file_until_dt1, until_dt).approximately_equal
+        assert ttl_duration1 == to_duration("1 hour")
+
+        # Provide different with_ttl - assure different key is generated.
+        until_dt = to_datetime().add(hours=1, minutes=1)
+        cache_file_key2, cache_file_until_dt2, ttl_duration2 = (
+            cache_file_store._generate_cache_file_key(key=key, with_ttl="1 hour 1 minute")
+        )
+        assert cache_file_key2 != cache_file_key
+        assert compare_datetimes(cache_file_until_dt2, until_dt).approximately_equal
+        assert ttl_duration2 == to_duration("1 hour 1 minute")
+
+    def test_get_file_path(self, cache_file_store):
+        """Test get file path from cache file object."""
+        cache_file = cache_file_store.create("test_file", mode="w+", suffix=".txt")
+        file_path = cache_file_store._get_file_path(cache_file)
+
+        assert file_path is not None
+
+    def test_until_datetime_by_options(self, cache_file_store):
+        """Test until datetime calculation based on options."""
+        now = to_datetime()
+
+        # Test with until_datetime
+        result, ttl_duration = cache_file_store._until_datetime_by_options(until_datetime=now)
+        assert result == now
+        assert ttl_duration is None
+
+        # -- From now on we expect a until_datetime in one hour
+        ttl_duration_expected = to_duration("1 hour")
+
+        # Test with with_ttl as timedelta
+        until_datetime_expected = to_datetime().add(hours=1)
+        ttl = timedelta(hours=1)
+        result, ttl_duration = cache_file_store._until_datetime_by_options(with_ttl=ttl)
+        assert compare_datetimes(result, until_datetime_expected).approximately_equal
+        assert ttl_duration == ttl_duration_expected
+
+        # Test with with_ttl as int (seconds)
+        until_datetime_expected = to_datetime().add(hours=1)
+        ttl_seconds = 3600
+        result, ttl_duration = cache_file_store._until_datetime_by_options(with_ttl=ttl_seconds)
+        assert compare_datetimes(result, until_datetime_expected).approximately_equal
+        assert ttl_duration == ttl_duration_expected
+
+        # Test with with_ttl as string ("1 hour")
+        until_datetime_expected = to_datetime().add(hours=1)
+        ttl_string = "1 hour"
+        result, ttl_duration = cache_file_store._until_datetime_by_options(with_ttl=ttl_string)
+        assert compare_datetimes(result, until_datetime_expected).approximately_equal
+        assert ttl_duration == ttl_duration_expected
+
+        # -- From now on we expect a until_datetime today at end of day
+        until_datetime_expected = to_datetime().end_of("day")
+        ttl_duration_expected = None
+
+        # Test default case (end of today)
+        result, ttl_duration = cache_file_store._until_datetime_by_options()
+        assert compare_datetimes(result, until_datetime_expected).equal
+        assert ttl_duration == ttl_duration_expected
+
+        # -- From now on we expect a until_datetime in one day at end of day
+        until_datetime_expected = to_datetime().add(days=1).end_of("day")
+        assert ttl_duration == ttl_duration_expected
+
+        # Test with until_date as date
+        until_date = date.today() + timedelta(days=1)
+        result, ttl_duration = cache_file_store._until_datetime_by_options(until_date=until_date)
+        assert compare_datetimes(result, until_datetime_expected).equal
+        assert ttl_duration == ttl_duration_expected
+
+        # -- Test with multiple options (until_datetime takes precedence)
+        specific_datetime = to_datetime().add(days=2)
+        result, ttl_duration = cache_file_store._until_datetime_by_options(
+            until_date=to_datetime().add(days=1).date(),
+            until_datetime=specific_datetime,
+            with_ttl=ttl,
+        )
+        assert compare_datetimes(result, specific_datetime).equal
+        assert ttl_duration is None
+
+        # Test with invalid inputs
+        with pytest.raises(ValueError):
+            cache_file_store._until_datetime_by_options(until_date="invalid-date")
+        with pytest.raises(ValueError):
+            cache_file_store._until_datetime_by_options(with_ttl="invalid-ttl")
+        with pytest.raises(ValueError):
+            cache_file_store._until_datetime_by_options(until_datetime="invalid-datetime")
+
+    def test_create_cache_file(self, cache_file_store):
+        """Test the creation of a cache file and ensure it is stored correctly."""
+        # Create a cache file for today's date
+        cache_file = cache_file_store.create("test_file", mode="w+", suffix=".txt")
+
+        # Check that the file exists in the store and is a file-like object
+        assert cache_file is not None
+        assert hasattr(cache_file, "name")
+        assert cache_file.name.endswith(".txt")
+
+        # Write some data to the file
+        cache_file.seek(0)
+        cache_file.write("Test data")
+        cache_file.seek(0)  # Reset file pointer
+        assert cache_file.read() == "Test data"
+
+    def test_get_cache_file(self, cache_file_store):
+        """Test retrieving an existing cache file by key."""
+        # Create a cache file and write data to it
+        cache_file = cache_file_store.create("test_file", mode="w+")
+        cache_file.seek(0)
+        cache_file.write("Test data")
+        cache_file.seek(0)
+
+        # Retrieve the cache file and verify the data
+        retrieved_file = cache_file_store.get("test_file")
+        assert retrieved_file is not None
+        retrieved_file.seek(0)
+        assert retrieved_file.read() == "Test data"
+
+    def test_set_custom_file_object(self, cache_file_store):
+        """Test setting a custom file-like object (BytesIO or StringIO) in the store."""
+        # Create a BytesIO object and set it into the cache
+        file_obj = io.BytesIO(b"Binary data")
+        cache_file_store.set("binary_file", file_obj)
+
+        # Retrieve the file from the store
+        retrieved_file = cache_file_store.get("binary_file")
+        assert isinstance(retrieved_file, io.BytesIO)
+        retrieved_file.seek(0)
+        assert retrieved_file.read() == b"Binary data"
+
+    def test_delete_cache_file(self, cache_file_store):
+        """Test deleting a cache file from the store."""
+        # Create multiple cache files
+        cache_file1 = cache_file_store.create("file1")
+        assert hasattr(cache_file1, "name")
+        cache_file2 = cache_file_store.create("file2")
+        assert hasattr(cache_file2, "name")
+
+        # Ensure the files are in the store
+        assert cache_file_store.get("file1") is cache_file1
+        assert cache_file_store.get("file2") is cache_file2
+
+        # Delete cache files
+        cache_file_store.delete("file1")
+        cache_file_store.delete("file2")
+
+        # Ensure the store is empty
+        assert cache_file_store.get("file1") is None
+        assert cache_file_store.get("file2") is None
+
+    def test_clear_all_cache_files(self, cache_file_store):
+        """Test clearing all cache files from the store."""
+        # Create multiple cache files
+        cache_file1 = cache_file_store.create("file1")
+        assert hasattr(cache_file1, "name")
+        cache_file2 = cache_file_store.create("file2")
+        assert hasattr(cache_file2, "name")
+
+        # Ensure the files are in the store
+        assert cache_file_store.get("file1") is cache_file1
+        assert cache_file_store.get("file2") is cache_file2
+
+        current_store = cache_file_store.current_store()
+        assert current_store != {}
+
+        # Clear all cache files
+        cache_file_store.clear(clear_all=True)
+
+        # Ensure the store is empty
+        assert cache_file_store.get("file1") is None
+        assert cache_file_store.get("file2") is None
+
+        current_store = cache_file_store.current_store()
+        assert current_store == {}
+
+    def test_clear_cache_files_by_date(self, cache_file_store):
+        """Test clearing cache files from the store by date."""
+        # Create multiple cache files
+        cache_file1 = cache_file_store.create("file1")
+        assert hasattr(cache_file1, "name")
+        cache_file2 = cache_file_store.create("file2")
+        assert hasattr(cache_file2, "name")
+
+        # Ensure the files are in the store
+        assert cache_file_store.get("file1") is cache_file1
+        assert cache_file_store.get("file2") is cache_file2
+
+        # Clear cache files that are older than today
+        cache_file_store.clear(before_datetime=to_datetime().start_of("day"))
+
+        # Ensure the files are in the store
+        assert cache_file_store.get("file1") is cache_file1
+        assert cache_file_store.get("file2") is cache_file2
+
+        # Clear cache files that are older than tomorrow
+        cache_file_store.clear(before_datetime=datetime.now() + timedelta(days=1))
+
+        # Ensure the store is empty
+        assert cache_file_store.get("file1") is None
+        assert cache_file_store.get("file2") is None
+
+    def test_cache_file_with_date(self, cache_file_store):
+        """Test creating and retrieving cache files with a specific date."""
+        # Use a specific date for cache file creation
+        specific_date = datetime(2023, 10, 10)
+        cache_file = cache_file_store.create("dated_file", mode="w+", until_date=specific_date)
+
+        # Write data to the cache file
+        cache_file.write("Dated data")
+        cache_file.seek(0)
+
+        # Retrieve the cache file with the specific date
+        retrieved_file = cache_file_store.get("dated_file", until_date=specific_date)
+        assert retrieved_file is not None
+        retrieved_file.seek(0)
+        assert retrieved_file.read() == "Dated data"
+
+    def test_recreate_existing_cache_file(self, cache_file_store):
+        """Test creating a cache file with an existing key does not overwrite the existing file."""
+        # Create a cache file
+        cache_file = cache_file_store.create("test_file", mode="w+")
+        cache_file.write("Original data")
+        cache_file.seek(0)
+
+        # Attempt to recreate the same file (should return the existing one)
+        new_file = cache_file_store.create("test_file")
+        assert new_file is cache_file  # Should be the same object
+        new_file.seek(0)
+        assert new_file.read() == "Original data"  # Data should be preserved
+
+        # Assure cache file store is a singleton
+        cache_file_store2 = CacheFileStore()
+        new_file = cache_file_store2.get("test_file")
+        assert new_file is cache_file  # Should be the same object
+
+    def test_cache_file_store_is_singleton(self, cache_file_store):
+        """Test re-creating a cache store provides the same store."""
+        # Create a cache file
+        cache_file = cache_file_store.create("test_file", mode="w+")
+        cache_file.write("Original data")
+        cache_file.seek(0)
+
+        # Assure cache file store is a singleton
+        cache_file_store2 = CacheFileStore()
+        new_file = cache_file_store2.get("test_file")
+        assert new_file is cache_file  # Should be the same object
+
+    def test_cache_file_store_save_store(self, cache_file_store):
+        # Creating a sample cache record
+        cache_file = MagicMock()
+        cache_file.name = "cache_file_path"
+        cache_file.mode = "wb+"
+        cache_record = CacheFileRecord(
+            cache_file=cache_file, until_datetime=to_datetime(), ttl_duration=None
+        )
+        cache_file_store._store = {"test_key": cache_record}
+
+        # Save the store to the file
+        cache_file_store.save_store()
+
+        # Verify the file content
+        with cache_file_store._store_file.open("r", encoding="utf-8", newline=None) as f:
+            store_loaded = json.load(f)
+            assert "test_key" in store_loaded
+            assert store_loaded["test_key"]["cache_file"] == "cache_file_path"
+            assert store_loaded["test_key"]["mode"] == "wb+"
+            assert store_loaded["test_key"]["until_datetime"] == to_datetime(
+                cache_record.until_datetime, as_string=True
+            )
+            assert store_loaded["test_key"]["ttl_duration"] is None
+
+    def test_cache_file_store_load_store(self, cache_file_store):
+        # Creating a sample cache record and save it to the file
+        cache_record = {
+            "test_key": {
+                "cache_file": "cache_file_path",
+                "mode": "wb+",
+                "until_datetime": to_datetime(as_string=True),
+                "ttl_duration": None,
+            }
+        }
+        with cache_file_store._store_file.open("w", encoding="utf-8", newline="\n") as f:
+            json.dump(cache_record, f, indent=4)
+
+        # Mock the open function to return a MagicMock for the cache file
+        with patch("builtins.open", new_callable=MagicMock) as mock_open:
+            mock_open.return_value.name = "cache_file_path"
+            mock_open.return_value.mode = "wb+"
+
+            # Load the store from the file
+            cache_file_store.load_store()
+
+            # Verify the loaded store
+            assert "test_key" in cache_file_store._store
+            loaded_record = cache_file_store._store["test_key"]
+            assert loaded_record.cache_file.name == "cache_file_path"
+            assert loaded_record.cache_file.mode == "wb+"
+            assert loaded_record.until_datetime == to_datetime(
+                cache_record["test_key"]["until_datetime"]
+            )
+            assert loaded_record.ttl_duration is None
+
+
+class TestCacheFileDecorators:
+    def test_cache_in_file_decorator_caches_function_result(self, cache_file_store):
+        """Test that the cache_in_file decorator caches a function result."""
+        # Clear store to assure it is empty
+        cache_file_store.clear(clear_all=True)
+        assert len(cache_file_store._store) == 0
+
+        # Define a simple function to decorate
+        @cache_in_file(mode="w+")
+        def my_function(until_date=None):
+            return "Some expensive computation result"
+
+        # Call the decorated function (should store result in cache)
+        result = my_function(until_date=datetime.now() + timedelta(days=1))
+        assert result == "Some expensive computation result"
+
+        # Assert that the create method was called to store the result
+        assert len(cache_file_store._store) == 1
+
+        # Check if the result was written to the cache file
+        key = next(iter(cache_file_store._store))
+        cache_file = cache_file_store._store[key].cache_file
+        assert cache_file is not None
+
+        # Assert correct content was written to the file
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == "Some expensive computation result"
+
+    def test_cache_in_file_decorator_uses_cache(self, cache_file_store):
+        """Test that the cache_in_file decorator reuses cached file on subsequent calls."""
+        # Clear store to assure it is empty
+        cache_file_store.clear(clear_all=True)
+        assert len(cache_file_store._store) == 0
+
+        # Define a simple function to decorate
+        @cache_in_file(mode="w+")
+        def my_function(until_date=None):
+            return "New result"
+
+        # Call the decorated function (should store result in cache)
+        result = my_function(until_date=to_datetime().add(days=1))
+        assert result == "New result"
+
+        # Assert result was written to cache file
+        key = next(iter(cache_file_store._store))
+        cache_file = cache_file_store._store[key].cache_file
+        assert cache_file is not None
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result
+
+        # Modify cache file
+        result2 = "Cached result"
+        cache_file.seek(0)
+        cache_file.write(result2)
+
+        # Call the decorated function again (should get result from cache)
+        result = my_function(until_date=to_datetime().add(days=1))
+        assert result == result2
+
+    def test_cache_in_file_decorator_forces_update_data(self, cache_file_store):
+        """Test that the cache_in_file decorator reuses cached file on subsequent calls."""
+        # Clear store to assure it is empty
+        cache_file_store.clear(clear_all=True)
+        assert len(cache_file_store._store) == 0
+
+        # Define a simple function to decorate
+        @cache_in_file(mode="w+")
+        def my_function(until_date=None):
+            return "New result"
+
+        until_date = to_datetime().add(days=1).date()
+
+        # Call the decorated function (should store result in cache)
+        result1 = "New result"
+        result = my_function(until_date=until_date)
+        assert result == result1
+
+        # Assert result was written to cache file
+        key = next(iter(cache_file_store._store))
+        cache_file = cache_file_store._store[key].cache_file
+        assert cache_file is not None
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result
+
+        # Modify cache file
+        result2 = "Cached result"
+        cache_file.seek(0)
+        cache_file.write(result2)
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result2
+
+        # Call the decorated function again with force update (should get result from function)
+        result = my_function(until_date=until_date, force_update=True)  # type: ignore[call-arg]
+        assert result == result1
+
+        # Assure result was written to the same cache file
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result1
+
+    def test_cache_in_file_handles_ttl(self, cache_file_store):
+        """Test that the cache_infile decorator handles the with_ttl parameter."""
+
+        # Define a simple function to decorate
+        @cache_in_file(mode="w+")
+        def my_function():
+            return "New result"
+
+        # Call the decorated function
+        result1 = my_function(with_ttl="1 second")  # type: ignore[call-arg]
+        assert result1 == "New result"
+        assert len(cache_file_store._store) == 1
+        key = list(cache_file_store._store.keys())[0]
+
+        # Assert result was written to cache file
+        key = next(iter(cache_file_store._store))
+        cache_file = cache_file_store._store[key].cache_file
+        assert cache_file is not None
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result1
+
+        # Modify cache file
+        result2 = "Cached result"
+        cache_file.seek(0)
+        cache_file.write(result2)
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result2
+
+        # Call the decorated function again
+        result = my_function(with_ttl="1 second")  # type: ignore[call-arg]
+        cache_file.seek(0)  # Move to the start of the file
+        assert cache_file.read() == result2
+        assert result == result2
+
+        # Wait one second to let the cache time out
+        sleep(2)
+
+        # Call again - cache should be timed out
+        result = my_function(with_ttl="1 second")  # type: ignore[call-arg]
+        assert result == result1
+
+    def test_cache_in_file_handles_bytes_return(self, cache_file_store):
+        """Test that the cache_infile decorator handles bytes returned from the function."""
+        # Clear store to assure it is empty
+        cache_file_store.clear(clear_all=True)
+        assert len(cache_file_store._store) == 0
+
+        # Define a function that returns bytes
+        @cache_in_file()
+        def my_function(until_date=None) -> bytes:
+            return b"Some binary data"
+
+        # Call the decorated function
+        result = my_function(until_date=datetime.now() + timedelta(days=1))
+
+        # Check if the binary data was written to the cache file
+        key = next(iter(cache_file_store._store))
+        cache_file = cache_file_store._store[key].cache_file
+        assert len(cache_file_store._store) == 1
+        assert cache_file is not None
+        cache_file.seek(0)
+        result1 = pickle.load(cache_file)
+        assert result1 == result
+
+        # Access cache
+        result = my_function(until_date=datetime.now() + timedelta(days=1))
+        assert len(cache_file_store._store) == 1
+        assert cache_file_store._store[key].cache_file is not None
+        assert result1 == result

tests/test_cacheutil.py

@@ -1,491 +0,0 @@
-"""Test Module for CacheFileStore Module."""
-
-import io
-import pickle
-from datetime import date, datetime, timedelta
-from time import sleep
-
-import pytest
-
-from akkudoktoreos.utils.cacheutil import CacheFileStore, cache_in_file
-from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime, to_duration
-
-# -----------------------------
-# CacheFileStore
-# -----------------------------
-
-
-@pytest.fixture
-def cache_store():
-    """A pytest fixture that creates a new CacheFileStore instance for testing."""
-    cache = CacheFileStore()
-    cache.clear(clear_all=True)
-    assert len(cache._store) == 0
-    return cache
-
-
-def test_generate_cache_file_key(cache_store):
-    """Test cache file key generation based on URL and date."""
-    key = "http://example.com"
-
-    # Provide until date - assure until_dt is used.
-    until_dt = to_datetime("2024-10-01")
-    cache_file_key, cache_file_until_dt, ttl_duration = cache_store._generate_cache_file_key(
-        key=key, until_datetime=until_dt
-    )
-    assert cache_file_key is not None
-    assert compare_datetimes(cache_file_until_dt, until_dt).equal
-
-    # Provide until date again - assure same key is generated.
-    cache_file_key1, cache_file_until_dt1, ttl_duration1 = cache_store._generate_cache_file_key(
-        key=key, until_datetime=until_dt
-    )
-    assert cache_file_key1 == cache_file_key
-    assert compare_datetimes(cache_file_until_dt1, until_dt).equal
-
-    # Provide no until date - assure today EOD is used.
-    no_until_dt = to_datetime().end_of("day")
-    cache_file_key, cache_file_until_dt, ttl_duration = cache_store._generate_cache_file_key(key)
-    assert cache_file_key is not None
-    assert compare_datetimes(cache_file_until_dt, no_until_dt).equal
-
-    # Provide with_ttl - assure until_dt is used.
-    until_dt = to_datetime().add(hours=1)
-    cache_file_key, cache_file_until_dt, ttl_duration = cache_store._generate_cache_file_key(
-        key, with_ttl="1 hour"
-    )
-    assert cache_file_key is not None
-    assert compare_datetimes(cache_file_until_dt, until_dt).approximately_equal
-    assert ttl_duration == to_duration("1 hour")
-
-    # Provide with_ttl again - assure same key is generated.
-    until_dt = to_datetime().add(hours=1)
-    cache_file_key1, cache_file_until_dt1, ttl_duration1 = cache_store._generate_cache_file_key(
-        key=key, with_ttl="1 hour"
-    )
-    assert cache_file_key1 == cache_file_key
-    assert compare_datetimes(cache_file_until_dt1, until_dt).approximately_equal
-    assert ttl_duration1 == to_duration("1 hour")
-
-    # Provide different with_ttl - assure different key is generated.
-    until_dt = to_datetime().add(hours=1, minutes=1)
-    cache_file_key2, cache_file_until_dt2, ttl_duration2 = cache_store._generate_cache_file_key(
-        key=key, with_ttl="1 hour 1 minute"
-    )
-    assert cache_file_key2 != cache_file_key
-    assert compare_datetimes(cache_file_until_dt2, until_dt).approximately_equal
-    assert ttl_duration2 == to_duration("1 hour 1 minute")
-
-
-def test_get_file_path(cache_store):
-    """Test get file path from cache file object."""
-    cache_file = cache_store.create("test_file", mode="w+", suffix=".txt")
-    file_path = cache_store._get_file_path(cache_file)
-
-    assert file_path is not None
-
-
-def test_until_datetime_by_options(cache_store):
-    """Test until datetime calculation based on options."""
-    now = to_datetime()
-
-    # Test with until_datetime
-    result, ttl_duration = cache_store._until_datetime_by_options(until_datetime=now)
-    assert result == now
-    assert ttl_duration is None
-
-    # -- From now on we expect a until_datetime in one hour
-    ttl_duration_expected = to_duration("1 hour")
-
-    # Test with with_ttl as timedelta
-    until_datetime_expected = to_datetime().add(hours=1)
-    ttl = timedelta(hours=1)
-    result, ttl_duration = cache_store._until_datetime_by_options(with_ttl=ttl)
-    assert compare_datetimes(result, until_datetime_expected).approximately_equal
-    assert ttl_duration == ttl_duration_expected
-
-    # Test with with_ttl as int (seconds)
-    until_datetime_expected = to_datetime().add(hours=1)
-    ttl_seconds = 3600
-    result, ttl_duration = cache_store._until_datetime_by_options(with_ttl=ttl_seconds)
-    assert compare_datetimes(result, until_datetime_expected).approximately_equal
-    assert ttl_duration == ttl_duration_expected
-
-    # Test with with_ttl as string ("1 hour")
-    until_datetime_expected = to_datetime().add(hours=1)
-    ttl_string = "1 hour"
-    result, ttl_duration = cache_store._until_datetime_by_options(with_ttl=ttl_string)
-    assert compare_datetimes(result, until_datetime_expected).approximately_equal
-    assert ttl_duration == ttl_duration_expected
-
-    # -- From now on we expect a until_datetime today at end of day
-    until_datetime_expected = to_datetime().end_of("day")
-    ttl_duration_expected = None
-
-    # Test default case (end of today)
-    result, ttl_duration = cache_store._until_datetime_by_options()
-    assert compare_datetimes(result, until_datetime_expected).equal
-    assert ttl_duration == ttl_duration_expected
-
-    # -- From now on we expect a until_datetime in one day at end of day
-    until_datetime_expected = to_datetime().add(days=1).end_of("day")
-    assert ttl_duration == ttl_duration_expected
-
-    # Test with until_date as date
-    until_date = date.today() + timedelta(days=1)
-    result, ttl_duration = cache_store._until_datetime_by_options(until_date=until_date)
-    assert compare_datetimes(result, until_datetime_expected).equal
-    assert ttl_duration == ttl_duration_expected
-
-    # -- Test with multiple options (until_datetime takes precedence)
-    specific_datetime = to_datetime().add(days=2)
-    result, ttl_duration = cache_store._until_datetime_by_options(
-        until_date=to_datetime().add(days=1).date(),
-        until_datetime=specific_datetime,
-        with_ttl=ttl,
-    )
-    assert compare_datetimes(result, specific_datetime).equal
-    assert ttl_duration is None
-
-    # Test with invalid inputs
-    with pytest.raises(ValueError):
-        cache_store._until_datetime_by_options(until_date="invalid-date")
-    with pytest.raises(ValueError):
-        cache_store._until_datetime_by_options(with_ttl="invalid-ttl")
-    with pytest.raises(ValueError):
-        cache_store._until_datetime_by_options(until_datetime="invalid-datetime")
-
-
-def test_create_cache_file(cache_store):
-    """Test the creation of a cache file and ensure it is stored correctly."""
-    # Create a cache file for today's date
-    cache_file = cache_store.create("test_file", mode="w+", suffix=".txt")
-
-    # Check that the file exists in the store and is a file-like object
-    assert cache_file is not None
-    assert hasattr(cache_file, "name")
-    assert cache_file.name.endswith(".txt")
-
-    # Write some data to the file
-    cache_file.seek(0)
-    cache_file.write("Test data")
-    cache_file.seek(0)  # Reset file pointer
-    assert cache_file.read() == "Test data"
-
-
-def test_get_cache_file(cache_store):
-    """Test retrieving an existing cache file by key."""
-    # Create a cache file and write data to it
-    cache_file = cache_store.create("test_file", mode="w+")
-    cache_file.seek(0)
-    cache_file.write("Test data")
-    cache_file.seek(0)
-
-    # Retrieve the cache file and verify the data
-    retrieved_file = cache_store.get("test_file")
-    assert retrieved_file is not None
-    retrieved_file.seek(0)
-    assert retrieved_file.read() == "Test data"
-
-
-def test_set_custom_file_object(cache_store):
-    """Test setting a custom file-like object (BytesIO or StringIO) in the store."""
-    # Create a BytesIO object and set it into the cache
-    file_obj = io.BytesIO(b"Binary data")
-    cache_store.set("binary_file", file_obj)
-
-    # Retrieve the file from the store
-    retrieved_file = cache_store.get("binary_file")
-    assert isinstance(retrieved_file, io.BytesIO)
-    retrieved_file.seek(0)
-    assert retrieved_file.read() == b"Binary data"
-
-
-def test_delete_cache_file(cache_store):
-    """Test deleting a cache file from the store."""
-    # Create multiple cache files
-    cache_file1 = cache_store.create("file1")
-    assert hasattr(cache_file1, "name")
-    cache_file2 = cache_store.create("file2")
-    assert hasattr(cache_file2, "name")
-
-    # Ensure the files are in the store
-    assert cache_store.get("file1") is cache_file1
-    assert cache_store.get("file2") is cache_file2
-
-    # Delete cache files
-    cache_store.delete("file1")
-    cache_store.delete("file2")
-
-    # Ensure the store is empty
-    assert cache_store.get("file1") is None
-    assert cache_store.get("file2") is None
-
-
-def test_clear_all_cache_files(cache_store):
-    """Test clearing all cache files from the store."""
-    # Create multiple cache files
-    cache_file1 = cache_store.create("file1")
-    assert hasattr(cache_file1, "name")
-    cache_file2 = cache_store.create("file2")
-    assert hasattr(cache_file2, "name")
-
-    # Ensure the files are in the store
-    assert cache_store.get("file1") is cache_file1
-    assert cache_store.get("file2") is cache_file2
-
-    # Clear all cache files
-    cache_store.clear(clear_all=True)
-
-    # Ensure the store is empty
-    assert cache_store.get("file1") is None
-    assert cache_store.get("file2") is None
-
-
-def test_clear_cache_files_by_date(cache_store):
-    """Test clearing cache files from the store by date."""
-    # Create multiple cache files
-    cache_file1 = cache_store.create("file1")
-    assert hasattr(cache_file1, "name")
-    cache_file2 = cache_store.create("file2")
-    assert hasattr(cache_file2, "name")
-
-    # Ensure the files are in the store
-    assert cache_store.get("file1") is cache_file1
-    assert cache_store.get("file2") is cache_file2
-
-    # Clear cache files that are older than today
-    cache_store.clear(before_datetime=to_datetime().start_of("day"))
-
-    # Ensure the files are in the store
-    assert cache_store.get("file1") is cache_file1
-    assert cache_store.get("file2") is cache_file2
-
-    # Clear cache files that are older than tomorrow
-    cache_store.clear(before_datetime=datetime.now() + timedelta(days=1))
-
-    # Ensure the store is empty
-    assert cache_store.get("file1") is None
-    assert cache_store.get("file2") is None
-
-
-def test_cache_file_with_date(cache_store):
-    """Test creating and retrieving cache files with a specific date."""
-    # Use a specific date for cache file creation
-    specific_date = datetime(2023, 10, 10)
-    cache_file = cache_store.create("dated_file", mode="w+", until_date=specific_date)
-
-    # Write data to the cache file
-    cache_file.write("Dated data")
-    cache_file.seek(0)
-
-    # Retrieve the cache file with the specific date
-    retrieved_file = cache_store.get("dated_file", until_date=specific_date)
-    assert retrieved_file is not None
-    retrieved_file.seek(0)
-    assert retrieved_file.read() == "Dated data"
-
-
-def test_recreate_existing_cache_file(cache_store):
-    """Test creating a cache file with an existing key does not overwrite the existing file."""
-    # Create a cache file
-    cache_file = cache_store.create("test_file", mode="w+")
-    cache_file.write("Original data")
-    cache_file.seek(0)
-
-    # Attempt to recreate the same file (should return the existing one)
-    new_file = cache_store.create("test_file")
-    assert new_file is cache_file  # Should be the same object
-    new_file.seek(0)
-    assert new_file.read() == "Original data"  # Data should be preserved
-
-    # Assure cache file store is a singleton
-    cache_store2 = CacheFileStore()
-    new_file = cache_store2.get("test_file")
-    assert new_file is cache_file  # Should be the same object
-
-
-def test_cache_store_is_singleton(cache_store):
-    """Test re-creating a cache store provides the same store."""
-    # Create a cache file
-    cache_file = cache_store.create("test_file", mode="w+")
-    cache_file.write("Original data")
-    cache_file.seek(0)
-
-    # Assure cache file store is a singleton
-    cache_store2 = CacheFileStore()
-    new_file = cache_store2.get("test_file")
-    assert new_file is cache_file  # Should be the same object
-
-
-def test_cache_in_file_decorator_caches_function_result(cache_store):
-    """Test that the cache_in_file decorator caches a function result."""
-    # Clear store to assure it is empty
-    cache_store.clear(clear_all=True)
-    assert len(cache_store._store) == 0
-
-    # Define a simple function to decorate
-    @cache_in_file(mode="w+")
-    def my_function(until_date=None):
-        return "Some expensive computation result"
-
-    # Call the decorated function (should store result in cache)
-    result = my_function(until_date=datetime.now() + timedelta(days=1))
-    assert result == "Some expensive computation result"
-
-    # Assert that the create method was called to store the result
-    assert len(cache_store._store) == 1
-
-    # Check if the result was written to the cache file
-    key = next(iter(cache_store._store))
-    cache_file = cache_store._store[key].cache_file
-    assert cache_file is not None
-
-    # Assert correct content was written to the file
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == "Some expensive computation result"
-
-
-def test_cache_in_file_decorator_uses_cache(cache_store):
-    """Test that the cache_in_file decorator reuses cached file on subsequent calls."""
-    # Clear store to assure it is empty
-    cache_store.clear(clear_all=True)
-    assert len(cache_store._store) == 0
-
-    # Define a simple function to decorate
-    @cache_in_file(mode="w+")
-    def my_function(until_date=None):
-        return "New result"
-
-    # Call the decorated function (should store result in cache)
-    result = my_function(until_date=to_datetime().add(days=1))
-    assert result == "New result"
-
-    # Assert result was written to cache file
-    key = next(iter(cache_store._store))
-    cache_file = cache_store._store[key].cache_file
-    assert cache_file is not None
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result
-
-    # Modify cache file
-    result2 = "Cached result"
-    cache_file.seek(0)
-    cache_file.write(result2)
-
-    # Call the decorated function again (should get result from cache)
-    result = my_function(until_date=to_datetime().add(days=1))
-    assert result == result2
-
-
-def test_cache_in_file_decorator_forces_update_data(cache_store):
-    """Test that the cache_in_file decorator reuses cached file on subsequent calls."""
-    # Clear store to assure it is empty
-    cache_store.clear(clear_all=True)
-    assert len(cache_store._store) == 0
-
-    # Define a simple function to decorate
-    @cache_in_file(mode="w+")
-    def my_function(until_date=None):
-        return "New result"
-
-    until_date = to_datetime().add(days=1).date()
-
-    # Call the decorated function (should store result in cache)
-    result1 = "New result"
-    result = my_function(until_date=until_date)
-    assert result == result1
-
-    # Assert result was written to cache file
-    key = next(iter(cache_store._store))
-    cache_file = cache_store._store[key].cache_file
-    assert cache_file is not None
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result
-
-    # Modify cache file
-    result2 = "Cached result"
-    cache_file.seek(0)
-    cache_file.write(result2)
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result2
-
-    # Call the decorated function again with force update (should get result from function)
-    result = my_function(until_date=until_date, force_update=True)  # type: ignore[call-arg]
-    assert result == result1
-
-    # Assure result was written to the same cache file
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result1
-
-
-def test_cache_in_file_handles_ttl(cache_store):
-    """Test that the cache_infile decorator handles the with_ttl parameter."""
-
-    # Define a simple function to decorate
-    @cache_in_file(mode="w+")
-    def my_function():
-        return "New result"
-
-    # Call the decorated function
-    result1 = my_function(with_ttl="1 second")  # type: ignore[call-arg]
-    assert result1 == "New result"
-    assert len(cache_store._store) == 1
-    key = list(cache_store._store.keys())[0]
-
-    # Assert result was written to cache file
-    key = next(iter(cache_store._store))
-    cache_file = cache_store._store[key].cache_file
-    assert cache_file is not None
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result1
-
-    # Modify cache file
-    result2 = "Cached result"
-    cache_file.seek(0)
-    cache_file.write(result2)
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result2
-
-    # Call the decorated function again
-    result = my_function(with_ttl="1 second")  # type: ignore[call-arg]
-    cache_file.seek(0)  # Move to the start of the file
-    assert cache_file.read() == result2
-    assert result == result2
-
-    # Wait one second to let the cache time out
-    sleep(2)
-
-    # Call again - cache should be timed out
-    result = my_function(with_ttl="1 second")  # type: ignore[call-arg]
-    assert result == result1
-
-
-def test_cache_in_file_handles_bytes_return(cache_store):
-    """Test that the cache_infile decorator handles bytes returned from the function."""
-    # Clear store to assure it is empty
-    cache_store.clear(clear_all=True)
-    assert len(cache_store._store) == 0
-
-    # Define a function that returns bytes
-    @cache_in_file()
-    def my_function(until_date=None) -> bytes:
-        return b"Some binary data"
-
-    # Call the decorated function
-    result = my_function(until_date=datetime.now() + timedelta(days=1))
-
-    # Check if the binary data was written to the cache file
-    key = next(iter(cache_store._store))
-    cache_file = cache_store._store[key].cache_file
-    assert len(cache_store._store) == 1
-    assert cache_file is not None
-    cache_file.seek(0)
-    result1 = pickle.load(cache_file)
-    assert result1 == result
-
-    # Access cache
-    result = my_function(until_date=datetime.now() + timedelta(days=1))
-    assert len(cache_store._store) == 1
-    assert cache_store._store[key].cache_file is not None
-    assert result1 == result

tests/test_class_ems.py

@@ -2,8 +2,8 @@ import numpy as np
 import pytest
 
 from akkudoktoreos.core.ems import (
-    EnergieManagementSystem,
-    EnergieManagementSystemParameters,
+    EnergyManagement,
+    EnergyManagementParameters,
     SimulationResult,
     get_ems,
 )
@@ -20,8 +20,8 @@ start_hour = 1
 
 # Example initialization of necessary components
 @pytest.fixture
-def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
-    """Fixture to create an EnergieManagementSystem instance with given test parameters."""
+def create_ems_instance(devices_eos, config_eos) -> EnergyManagement:
+    """Fixture to create an EnergyManagement instance with given test parameters."""
     # Assure configuration holds the correct values
     config_eos.merge_settings_from_dict(
         {"prediction": {"hours": 48}, "optimization": {"hours": 24}}
@@ -227,7 +227,7 @@ def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
     # Initialize the energy management system with the respective parameters
     ems = get_ems()
     ems.set_parameters(
-        EnergieManagementSystemParameters(
+        EnergyManagementParameters(
             pv_prognose_wh=pv_prognose_wh,
             strompreis_euro_pro_wh=strompreis_euro_pro_wh,
             einspeiseverguetung_euro_pro_wh=einspeiseverguetung_euro_pro_wh,
@@ -243,7 +243,7 @@ def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
 
 
 def test_simulation(create_ems_instance):
-    """Test the EnergieManagementSystem simulation method."""
+    """Test the EnergyManagement simulation method."""
     ems = create_ems_instance
 
     # Simulate starting from hour 1 (this value can be adjusted)

tests/test_class_ems_2.py

@@ -2,8 +2,8 @@ import numpy as np
 import pytest
 
 from akkudoktoreos.core.ems import (
-    EnergieManagementSystem,
-    EnergieManagementSystemParameters,
+    EnergyManagement,
+    EnergyManagementParameters,
     SimulationResult,
     get_ems,
 )
@@ -20,8 +20,8 @@ start_hour = 0
 
 # Example initialization of necessary components
 @pytest.fixture
-def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
-    """Fixture to create an EnergieManagementSystem instance with given test parameters."""
+def create_ems_instance(devices_eos, config_eos) -> EnergyManagement:
+    """Fixture to create an EnergyManagement instance with given test parameters."""
     # Assure configuration holds the correct values
     config_eos.merge_settings_from_dict(
         {"prediction": {"hours": 48}, "optimization": {"hours": 24}}
@@ -130,7 +130,7 @@ def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
     # Initialize the energy management system with the respective parameters
     ems = get_ems()
     ems.set_parameters(
-        EnergieManagementSystemParameters(
+        EnergyManagementParameters(
             pv_prognose_wh=pv_prognose_wh,
             strompreis_euro_pro_wh=strompreis_euro_pro_wh,
             einspeiseverguetung_euro_pro_wh=einspeiseverguetung_euro_pro_wh,
@@ -153,7 +153,7 @@ def create_ems_instance(devices_eos, config_eos) -> EnergieManagementSystem:
 
 
 def test_simulation(create_ems_instance):
-    """Test the EnergieManagementSystem simulation method."""
+    """Test the EnergyManagement simulation method."""
     ems = create_ems_instance
 
     # Simulate starting from hour 0 (this value can be adjusted)
@@ -254,7 +254,7 @@ def test_simulation(create_ems_instance):
 
 
 def test_set_parameters(create_ems_instance):
-    """Test the set_parameters method of EnergieManagementSystem."""
+    """Test the set_parameters method of EnergyManagement."""
     ems = create_ems_instance
 
     # Check if parameters are set correctly
@@ -267,7 +267,7 @@ def test_set_parameters(create_ems_instance):
 
 
 def test_set_akku_discharge_hours(create_ems_instance):
-    """Test the set_akku_discharge_hours method of EnergieManagementSystem."""
+    """Test the set_akku_discharge_hours method of EnergyManagement."""
     ems = create_ems_instance
     discharge_hours = np.full(ems.config.prediction.hours, 1.0)
     ems.set_akku_discharge_hours(discharge_hours)
@@ -277,7 +277,7 @@ def test_set_akku_discharge_hours(create_ems_instance):
 
 
 def test_set_akku_ac_charge_hours(create_ems_instance):
-    """Test the set_akku_ac_charge_hours method of EnergieManagementSystem."""
+    """Test the set_akku_ac_charge_hours method of EnergyManagement."""
     ems = create_ems_instance
     ac_charge_hours = np.full(ems.config.prediction.hours, 1.0)
     ems.set_akku_ac_charge_hours(ac_charge_hours)
@@ -287,7 +287,7 @@ def test_set_akku_ac_charge_hours(create_ems_instance):
 
 
 def test_set_akku_dc_charge_hours(create_ems_instance):
-    """Test the set_akku_dc_charge_hours method of EnergieManagementSystem."""
+    """Test the set_akku_dc_charge_hours method of EnergyManagement."""
     ems = create_ems_instance
     dc_charge_hours = np.full(ems.config.prediction.hours, 1.0)
     ems.set_akku_dc_charge_hours(dc_charge_hours)
@@ -297,7 +297,7 @@ def test_set_akku_dc_charge_hours(create_ems_instance):
 
 
 def test_set_ev_charge_hours(create_ems_instance):
-    """Test the set_ev_charge_hours method of EnergieManagementSystem."""
+    """Test the set_ev_charge_hours method of EnergyManagement."""
     ems = create_ems_instance
     ev_charge_hours = np.full(ems.config.prediction.hours, 1.0)
     ems.set_ev_charge_hours(ev_charge_hours)
@@ -307,7 +307,7 @@ def test_set_ev_charge_hours(create_ems_instance):
 
 
 def test_reset(create_ems_instance):
-    """Test the reset method of EnergieManagementSystem."""
+    """Test the reset method of EnergyManagement."""
     ems = create_ems_instance
     ems.reset()
     assert ems.ev.current_soc_percentage() == 100, "EV SOC should be reset to initial value"
@@ -317,7 +317,7 @@ def test_reset(create_ems_instance):
 
 
 def test_simulate_start_now(create_ems_instance):
-    """Test the simulate_start_now method of EnergieManagementSystem."""
+    """Test the simulate_start_now method of EnergyManagement."""
     ems = create_ems_instance
     result = ems.simulate_start_now()
     assert result is not None, "Result should not be None"

tests/test_class_optimize.py

@@ -86,7 +86,8 @@ def test_optimize(
             parameters=input_data, start_hour=start_hour, ngen=ngen
         )
         # Write test output to file, so we can take it as new data on intended change
-        with open(DIR_TESTDATA / f"new_{fn_out}", "w") as f_out:
+        TESTDATA_FILE = DIR_TESTDATA / f"new_{fn_out}"
+        with TESTDATA_FILE.open("w", encoding="utf-8", newline="\n") as f_out:
             f_out.write(ergebnis.model_dump_json(indent=4, exclude_unset=True))
 
         assert ergebnis.result.Gesamtbilanz_Euro == pytest.approx(

tests/test_config.py

@@ -1,5 +1,6 @@
 import tempfile
 from pathlib import Path
+from typing import Union
 from unittest.mock import patch
 
 import pytest
@@ -46,12 +47,19 @@ def test_computed_paths(config_eos):
                 "general": {
                     "data_folder_path": "/base/data",
                     "data_output_subpath": "extra/output",
-                    "data_cache_subpath": "somewhere/cache",
-                }
+                },
+                "cache": {
+                    "subpath": "somewhere/cache",
+                },
             }
         )
+    assert config_eos.general.data_folder_path == Path("/base/data")
     assert config_eos.general.data_output_path == Path("/base/data/extra/output")
-    assert config_eos.general.data_cache_path == Path("/base/data/somewhere/cache")
+    assert config_eos.cache.path() == Path("/base/data/somewhere/cache")
+    # Check non configurable pathes
+    assert config_eos.package_root_path == Path(__file__).parent.parent.resolve().joinpath(
+        "src/akkudoktoreos"
+    )
     # reset settings so the config_eos fixture can verify the default paths
     config_eos.reset_settings()
 
@@ -374,3 +382,64 @@ def test_get_nested_key(path, expected_value, exception, config_eos):
     else:
         with pytest.raises(exception):
             config_eos.get_config_value(path)
+
+
+def test_merge_settings_from_dict_invalid(config_eos):
+    """Test merging invalid data."""
+    invalid_settings = {
+        "general": {
+            "latitude": "invalid_latitude"  # Should be a float
+        },
+    }
+
+    with pytest.raises(Exception):  # Pydantic ValidationError expected
+        config_eos.merge_settings_from_dict(invalid_settings)
+
+
+def test_merge_settings_partial(config_eos):
+    """Test merging only a subset of settings."""
+    partial_settings: dict[str, dict[str, Union[float, None, str]]] = {
+        "general": {
+            "latitude": 51.1657  # Only latitude is updated
+        },
+    }
+
+    config_eos.merge_settings_from_dict(partial_settings)
+    assert config_eos.general.latitude == 51.1657
+    assert config_eos.general.longitude == 13.405  # Should remain unchanged
+
+    partial_settings = {
+        "weather": {
+            "provider": "BrightSky",
+        },
+    }
+
+    config_eos.merge_settings_from_dict(partial_settings)
+    assert config_eos.weather.provider == "BrightSky"
+
+    partial_settings = {
+        "general": {
+            "latitude": None,
+        },
+        "weather": {
+            "provider": "ClearOutside",
+        },
+    }
+
+    config_eos.merge_settings_from_dict(partial_settings)
+    assert config_eos.general.latitude is None
+    assert config_eos.weather.provider == "ClearOutside"
+
+    # Assure update keeps same values
+    config_eos.update()
+    assert config_eos.general.latitude is None
+    assert config_eos.weather.provider == "ClearOutside"
+
+
+def test_merge_settings_empty(config_eos):
+    """Test merging an empty dictionary does not change settings."""
+    original_latitude = config_eos.general.latitude
+
+    config_eos.merge_settings_from_dict({})  # No changes
+
+    assert config_eos.general.latitude == original_latitude  # Should remain unchanged

tests/test_dataabc.py

@@ -562,6 +562,102 @@ class TestDataSequence:
         assert dates == [to_datetime(datetime(2023, 11, 5)), to_datetime(datetime(2023, 11, 6))]
         assert values == [0.8, 0.9]
 
+    def test_to_dataframe_full_data(self, sequence):
+        """Test conversion of all records to a DataFrame without filtering."""
+        record1 = self.create_test_record("2024-01-01T12:00:00Z", 10)
+        record2 = self.create_test_record("2024-01-01T13:00:00Z", 20)
+        record3 = self.create_test_record("2024-01-01T14:00:00Z", 30)
+        sequence.append(record1)
+        sequence.append(record2)
+        sequence.append(record3)
+
+        df = sequence.to_dataframe()
+
+        # Validate DataFrame structure
+        assert isinstance(df, pd.DataFrame)
+        assert not df.empty
+        assert len(df) == 3  # All records should be included
+        assert "data_value" in df.columns
+
+    def test_to_dataframe_with_filter(self, sequence):
+        """Test filtering records by datetime range."""
+        record1 = self.create_test_record("2024-01-01T12:00:00Z", 10)
+        record2 = self.create_test_record("2024-01-01T13:00:00Z", 20)
+        record3 = self.create_test_record("2024-01-01T14:00:00Z", 30)
+        sequence.append(record1)
+        sequence.append(record2)
+        sequence.append(record3)
+
+        start = to_datetime("2024-01-01T12:30:00Z")
+        end = to_datetime("2024-01-01T14:00:00Z")
+
+        df = sequence.to_dataframe(start_datetime=start, end_datetime=end)
+
+        assert isinstance(df, pd.DataFrame)
+        assert not df.empty
+        assert len(df) == 1  # Only one record should match the range
+        assert df.index[0] == pd.Timestamp("2024-01-01T13:00:00Z")
+
+    def test_to_dataframe_no_matching_records(self, sequence):
+        """Test when no records match the given datetime filter."""
+        record1 = self.create_test_record("2024-01-01T12:00:00Z", 10)
+        record2 = self.create_test_record("2024-01-01T13:00:00Z", 20)
+        sequence.append(record1)
+        sequence.append(record2)
+
+        start = to_datetime("2024-01-01T14:00:00Z")  # Start time after all records
+        end = to_datetime("2024-01-01T15:00:00Z")
+
+        df = sequence.to_dataframe(start_datetime=start, end_datetime=end)
+
+        assert isinstance(df, pd.DataFrame)
+        assert df.empty  # No records should match
+
+    def test_to_dataframe_empty_sequence(self, sequence):
+        """Test when DataSequence has no records."""
+        sequence = DataSequence(records=[])
+
+        df = sequence.to_dataframe()
+
+        assert isinstance(df, pd.DataFrame)
+        assert df.empty  # Should return an empty DataFrame
+
+    def test_to_dataframe_no_start_datetime(self, sequence):
+        """Test when only end_datetime is given (all past records should be included)."""
+        record1 = self.create_test_record("2024-01-01T12:00:00Z", 10)
+        record2 = self.create_test_record("2024-01-01T13:00:00Z", 20)
+        record3 = self.create_test_record("2024-01-01T14:00:00Z", 30)
+        sequence.append(record1)
+        sequence.append(record2)
+        sequence.append(record3)
+
+        end = to_datetime("2024-01-01T13:00:00Z")  # Include only first record
+
+        df = sequence.to_dataframe(end_datetime=end)
+
+        assert isinstance(df, pd.DataFrame)
+        assert not df.empty
+        assert len(df) == 1
+        assert df.index[0] == pd.Timestamp("2024-01-01T12:00:00Z")
+
+    def test_to_dataframe_no_end_datetime(self, sequence):
+        """Test when only start_datetime is given (all future records should be included)."""
+        record1 = self.create_test_record("2024-01-01T12:00:00Z", 10)
+        record2 = self.create_test_record("2024-01-01T13:00:00Z", 20)
+        record3 = self.create_test_record("2024-01-01T14:00:00Z", 30)
+        sequence.append(record1)
+        sequence.append(record2)
+        sequence.append(record3)
+
+        start = to_datetime("2024-01-01T13:00:00Z")  # Include last two records
+
+        df = sequence.to_dataframe(start_datetime=start)
+
+        assert isinstance(df, pd.DataFrame)
+        assert not df.empty
+        assert len(df) == 2
+        assert df.index[0] == pd.Timestamp("2024-01-01T13:00:00Z")
+
 
 class TestDataProvider:
     # Fixtures and helper functions

tests/test_doc.py

@@ -1,4 +1,5 @@
 import json
+import os
 import sys
 from pathlib import Path
 from unittest.mock import patch
@@ -14,7 +15,7 @@ def test_openapi_spec_current(config_eos):
     expected_spec_path = DIR_PROJECT_ROOT / "openapi.json"
     new_spec_path = DIR_TESTDATA / "openapi-new.json"
 
-    with open(expected_spec_path) as f_expected:
+    with expected_spec_path.open("r", encoding="utf-8", newline=None) as f_expected:
         expected_spec = json.load(f_expected)
 
     # Patch get_config and import within guard to patch global variables within the eos module.
@@ -25,12 +26,14 @@ def test_openapi_spec_current(config_eos):
         from scripts import generate_openapi
 
         spec = generate_openapi.generate_openapi()
+        spec_str = json.dumps(spec, indent=4, sort_keys=True)
 
-    with open(new_spec_path, "w") as f_new:
-        json.dump(spec, f_new, indent=4, sort_keys=True)
+    if os.name == "nt":
+        spec_str = spec_str.replace("127.0.0.1", "0.0.0.0")
+    with new_spec_path.open("w", encoding="utf-8", newline="\n") as f_new:
+        f_new.write(spec_str)
 
     # Serialize to ensure comparison is consistent
-    spec_str = json.dumps(spec, indent=4, sort_keys=True)
     expected_spec_str = json.dumps(expected_spec, indent=4, sort_keys=True)
 
     try:
@@ -47,7 +50,7 @@ def test_openapi_md_current(config_eos):
     expected_spec_md_path = DIR_PROJECT_ROOT / "docs" / "_generated" / "openapi.md"
     new_spec_md_path = DIR_TESTDATA / "openapi-new.md"
 
-    with open(expected_spec_md_path, encoding="utf8") as f_expected:
+    with expected_spec_md_path.open("r", encoding="utf-8", newline=None) as f_expected:
         expected_spec_md = f_expected.read()
 
     # Patch get_config and import within guard to patch global variables within the eos module.
@@ -59,7 +62,9 @@ def test_openapi_md_current(config_eos):
 
         spec_md = generate_openapi_md.generate_openapi_md()
 
-    with open(new_spec_md_path, "w", encoding="utf8") as f_new:
+    if os.name == "nt":
+        spec_md = spec_md.replace("127.0.0.1", "0.0.0.0")
+    with new_spec_md_path.open("w", encoding="utf-8", newline="\n") as f_new:
         f_new.write(spec_md)
 
     try:
@@ -76,7 +81,7 @@ def test_config_md_current(config_eos):
     expected_config_md_path = DIR_PROJECT_ROOT / "docs" / "_generated" / "config.md"
     new_config_md_path = DIR_TESTDATA / "config-new.md"
 
-    with open(expected_config_md_path, encoding="utf8") as f_expected:
+    with expected_config_md_path.open("r", encoding="utf-8", newline=None) as f_expected:
         expected_config_md = f_expected.read()
 
     # Patch get_config and import within guard to patch global variables within the eos module.
@@ -88,7 +93,9 @@ def test_config_md_current(config_eos):
 
         config_md = generate_config_md.generate_config_md(config_eos)
 
-    with open(new_config_md_path, "w", encoding="utf8") as f_new:
+    if os.name == "nt":
+        config_md = config_md.replace("127.0.0.1", "0.0.0.0").replace("\\\\", "/")
+    with new_config_md_path.open("w", encoding="utf-8", newline="\n") as f_new:
         f_new.write(config_md)
 
     try:

tests/test_elecpriceakkudoktor.py

@@ -6,6 +6,7 @@ import numpy as np
 import pytest
 import requests
 
+from akkudoktoreos.core.cache import CacheFileStore
 from akkudoktoreos.core.ems import get_ems
 from akkudoktoreos.core.logging import get_logger
 from akkudoktoreos.prediction.elecpriceakkudoktor import (
@@ -13,7 +14,6 @@ from akkudoktoreos.prediction.elecpriceakkudoktor import (
     AkkudoktorElecPriceValue,
     ElecPriceAkkudoktor,
 )
-from akkudoktoreos.utils.cacheutil import CacheFileStore
 from akkudoktoreos.utils.datetimeutil import to_datetime
 
 DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata")
@@ -36,7 +36,9 @@ def provider(monkeypatch, config_eos):
 @pytest.fixture
 def sample_akkudoktor_1_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_ELECPRICEAKKUDOKTOR_1_JSON, "r") as f_res:
+    with FILE_TESTDATA_ELECPRICEAKKUDOKTOR_1_JSON.open(
+        "r", encoding="utf-8", newline=None
+    ) as f_res:
         input_data = json.load(f_res)
     return input_data
 
@@ -173,7 +175,7 @@ def test_request_forecast_status_codes(
         provider._request_forecast()
 
 
-@patch("akkudoktoreos.utils.cacheutil.CacheFileStore")
+@patch("akkudoktoreos.core.cache.CacheFileStore")
 def test_cache_integration(mock_cache, provider):
     """Test caching of 8-day electricity price data."""
     mock_cache_instance = mock_cache.return_value
@@ -208,5 +210,7 @@ def test_akkudoktor_development_forecast_data(provider):
 
     akkudoktor_data = provider._request_forecast()
 
-    with open(FILE_TESTDATA_ELECPRICEAKKUDOKTOR_1_JSON, "w") as f_out:
+    with FILE_TESTDATA_ELECPRICEAKKUDOKTOR_1_JSON.open(
+        "w", encoding="utf-8", newline="\n"
+    ) as f_out:
         json.dump(akkudoktor_data, f_out, indent=4)

tests/test_elecpriceimport.py

@@ -33,7 +33,7 @@ def provider(sample_import_1_json, config_eos):
 @pytest.fixture
 def sample_import_1_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_ELECPRICEIMPORT_1_JSON, "r") as f_res:
+    with FILE_TESTDATA_ELECPRICEIMPORT_1_JSON.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = json.load(f_res)
     return input_data
 

tests/test_eosdashserver.py

@@ -0,0 +1,51 @@
+import time
+from http import HTTPStatus
+
+import requests
+
+
+class TestEOSDash:
+    def test_eosdash_started(self, server_setup_for_class, is_system_test):
+        """Test the EOSdash server is started by EOS server."""
+        server = server_setup_for_class["server"]
+        eosdash_server = server_setup_for_class["eosdash_server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+        timeout = server_setup_for_class["timeout"]
+
+        # Assure EOSdash is up
+        startup = False
+        error = ""
+        for retries in range(int(timeout / 3)):
+            try:
+                result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+                if result.status_code == HTTPStatus.OK:
+                    startup = True
+                    break
+                error = f"{result.status_code}, {str(result.content)}"
+            except Exception as ex:
+                error = str(ex)
+            time.sleep(3)
+        assert startup, f"Connection to {eosdash_server}/eosdash/health failed: {error}"
+        assert result.json()["status"] == "alive"
+
+    def test_eosdash_proxied_by_eos(self, server_setup_for_class, is_system_test):
+        """Test the EOSdash server proxied by EOS server."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+        timeout = server_setup_for_class["timeout"]
+
+        # Assure EOSdash is up
+        startup = False
+        error = ""
+        for retries in range(int(timeout / 3)):
+            try:
+                result = requests.get(f"{server}/eosdash/health", timeout=2)
+                if result.status_code == HTTPStatus.OK:
+                    startup = True
+                    break
+                error = f"{result.status_code}, {str(result.content)}"
+            except Exception as ex:
+                error = str(ex)
+            time.sleep(3)
+        assert startup, f"Connection to {server}/eosdash/health failed: {error}"
+        assert result.json()["status"] == "alive"

tests/test_loadakkudoktor.py

@@ -26,6 +26,8 @@ def provider(config_eos):
         }
     }
     config_eos.merge_settings_from_dict(settings)
+    assert config_eos.load.provider == "LoadAkkudoktor"
+    assert config_eos.load.provider_settings.loadakkudoktor_year_energy == 1000
     return LoadAkkudoktor()
 
 

tests/test_logging.py

@@ -1,8 +1,8 @@
 """Test Module for logging Module."""
 
 import logging
-import os
 from logging.handlers import RotatingFileHandler
+from pathlib import Path
 
 import pytest
 
@@ -13,16 +13,7 @@ from akkudoktoreos.core.logging import get_logger
 # -----------------------------
 
 
-@pytest.fixture
-def clean_up_log_file():
-    """Fixture to clean up log files after tests."""
-    log_file = "test.log"
-    yield log_file
-    if os.path.exists(log_file):
-        os.remove(log_file)
-
-
-def test_get_logger_console_logging(clean_up_log_file):
+def test_get_logger_console_logging():
     """Test logger creation with console logging."""
     logger = get_logger("test_logger", logging_level="DEBUG")
 
@@ -37,9 +28,10 @@ def test_get_logger_console_logging(clean_up_log_file):
     assert isinstance(logger.handlers[0], logging.StreamHandler)
 
 
-def test_get_logger_file_logging(clean_up_log_file):
+def test_get_logger_file_logging(tmpdir):
     """Test logger creation with file logging."""
-    logger = get_logger("test_logger", log_file="test.log", logging_level="WARNING")
+    log_file = Path(tmpdir).joinpath("test.log")
+    logger = get_logger("test_logger", log_file=str(log_file), logging_level="WARNING")
 
     # Check logger name
     assert logger.name == "test_logger"
@@ -53,10 +45,10 @@ def test_get_logger_file_logging(clean_up_log_file):
     assert isinstance(logger.handlers[1], RotatingFileHandler)
 
     # Check file existence
-    assert os.path.exists("test.log")
+    assert log_file.exists()
 
 
-def test_get_logger_no_file_logging(clean_up_log_file):
+def test_get_logger_no_file_logging():
     """Test logger creation without file logging."""
     logger = get_logger("test_logger")
 
@@ -71,7 +63,7 @@ def test_get_logger_no_file_logging(clean_up_log_file):
     assert isinstance(logger.handlers[0], logging.StreamHandler)
 
 
-def test_get_logger_with_invalid_level(clean_up_log_file):
+def test_get_logger_with_invalid_level():
     """Test logger creation with an invalid logging level."""
     with pytest.raises(ValueError, match="Unknown loggin level: INVALID"):
         logger = get_logger("test_logger", logging_level="INVALID")

tests/test_predictionabc.py

@@ -201,8 +201,8 @@ class TestPredictionProvider:
     def test_update_method_force_enable(self, provider, monkeypatch):
         """Test that `update` executes when `force_enable` is True, even if `enabled` is False."""
         # Preset values that are needed by update
-        monkeypatch.setenv("EOS_PREDICTION__LATITUDE", "37.7749")
-        monkeypatch.setenv("EOS_PREDICTION__LONGITUDE", "-122.4194")
+        monkeypatch.setenv("EOS_GENERAL__LATITUDE", "37.7749")
+        monkeypatch.setenv("EOS_GENERAL__LONGITUDE", "-122.4194")
 
         # Override enabled to return False for this test
         DerivedPredictionProvider.provider_enabled = False

tests/test_pvforecastakkudoktor.py

@@ -80,7 +80,7 @@ def sample_settings(config_eos):
 @pytest.fixture
 def sample_forecast_data():
     """Fixture that returns sample forecast data converted to pydantic model."""
-    with open(FILE_TESTDATA_PV_FORECAST_INPUT_1, "r", encoding="utf8") as f_in:
+    with FILE_TESTDATA_PV_FORECAST_INPUT_1.open("r", encoding="utf-8", newline=None) as f_in:
         input_data = f_in.read()
     return PVForecastAkkudoktor._validate_data(input_data)
 
@@ -88,7 +88,7 @@ def sample_forecast_data():
 @pytest.fixture
 def sample_forecast_data_raw():
     """Fixture that returns raw sample forecast data."""
-    with open(FILE_TESTDATA_PV_FORECAST_INPUT_1, "r", encoding="utf8") as f_in:
+    with FILE_TESTDATA_PV_FORECAST_INPUT_1.open("r", encoding="utf-8", newline=None) as f_in:
         input_data = f_in.read()
     return input_data
 
@@ -96,7 +96,7 @@ def sample_forecast_data_raw():
 @pytest.fixture
 def sample_forecast_report():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_PV_FORECAST_RESULT_1, "r", encoding="utf8") as f_res:
+    with FILE_TESTDATA_PV_FORECAST_RESULT_1.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = f_res.read()
     return input_data
 

tests/test_pvforecastimport.py

@@ -33,7 +33,7 @@ def provider(sample_import_1_json, config_eos):
 @pytest.fixture
 def sample_import_1_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_PVFORECASTIMPORT_1_JSON, "r") as f_res:
+    with FILE_TESTDATA_PVFORECASTIMPORT_1_JSON.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = json.load(f_res)
     return input_data
 

tests/test_server.py

@@ -1,13 +1,443 @@
+import json
+import os
+import signal
+import time
 from http import HTTPStatus
+from pathlib import Path
 
+import psutil
+import pytest
 import requests
 
+DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata")
 
-def test_server(server, config_eos):
-    """Test the server."""
-    # validate correct path in server
-    assert config_eos.general.data_folder_path is not None
-    assert config_eos.general.data_folder_path.is_dir()
+FILE_TESTDATA_EOSSERVER_CONFIG_1 = DIR_TESTDATA.joinpath("eosserver_config_1.json")
 
-    result = requests.get(f"{server}/v1/config")
-    assert result.status_code == HTTPStatus.OK
+
+class TestServer:
+    def test_server_setup_for_class(self, server_setup_for_class):
+        """Ensure server is started."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        result = requests.get(f"{server}/v1/config")
+        assert result.status_code == HTTPStatus.OK
+
+        # Get testing config
+        config_json = result.json()
+        config_folder_path = Path(config_json["general"]["config_folder_path"])
+        config_file_path = Path(config_json["general"]["config_file_path"])
+        data_folder_path = Path(config_json["general"]["data_folder_path"])
+        data_ouput_path = Path(config_json["general"]["data_output_path"])
+        # Assure we are working in test environment
+        assert str(config_folder_path).startswith(eos_dir)
+        assert str(config_file_path).startswith(eos_dir)
+        assert str(data_folder_path).startswith(eos_dir)
+        assert str(data_ouput_path).startswith(eos_dir)
+
+    def test_prediction_brightsky(self, server_setup_for_class, is_system_test):
+        """Test weather prediction by BrightSky."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        result = requests.get(f"{server}/v1/config")
+        assert result.status_code == HTTPStatus.OK
+
+        # Get testing config
+        config_json = result.json()
+        config_folder_path = Path(config_json["general"]["config_folder_path"])
+        # Assure we are working in test environment
+        assert str(config_folder_path).startswith(eos_dir)
+
+        result = requests.put(f"{server}/v1/config/weather/provider", json="BrightSky")
+        assert result.status_code == HTTPStatus.OK
+
+        # Assure prediction is enabled
+        result = requests.get(f"{server}/v1/prediction/providers?enabled=true")
+        assert result.status_code == HTTPStatus.OK
+        providers = result.json()
+        assert "BrightSky" in providers
+
+        if is_system_test:
+            result = requests.post(f"{server}/v1/prediction/update/BrightSky")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air")
+            assert result.status_code == HTTPStatus.OK
+
+            data = result.json()
+            assert len(data["data"]) > 24
+
+        else:
+            pass
+
+    def test_prediction_clearoutside(self, server_setup_for_class, is_system_test):
+        """Test weather prediction by ClearOutside."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        result = requests.put(f"{server}/v1/config/weather/provider", json="ClearOutside")
+        assert result.status_code == HTTPStatus.OK
+
+        # Assure prediction is enabled
+        result = requests.get(f"{server}/v1/prediction/providers?enabled=true")
+        assert result.status_code == HTTPStatus.OK
+        providers = result.json()
+        assert "ClearOutside" in providers
+
+        if is_system_test:
+            result = requests.post(f"{server}/v1/prediction/update/ClearOutside")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air")
+            assert result.status_code == HTTPStatus.OK
+
+            data = result.json()
+            assert len(data["data"]) > 24
+
+        else:
+            pass
+
+    def test_prediction_pvforecastakkudoktor(self, server_setup_for_class, is_system_test):
+        """Test PV prediction by PVForecastAkkudoktor."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        # Reset config
+        with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd:
+            config = json.load(fd)
+        config["pvforecast"]["provider"] = "PVForecastAkkudoktor"
+        result = requests.put(f"{server}/v1/config", json=config)
+        assert result.status_code == HTTPStatus.OK
+
+        # Assure prediction is enabled
+        result = requests.get(f"{server}/v1/prediction/providers?enabled=true")
+        assert result.status_code == HTTPStatus.OK
+        providers = result.json()
+        assert "PVForecastAkkudoktor" in providers
+
+        if is_system_test:
+            result = requests.post(f"{server}/v1/prediction/update/PVForecastAkkudoktor")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=pvforecast_ac_power")
+            assert result.status_code == HTTPStatus.OK
+
+            data = result.json()
+            assert len(data["data"]) > 24
+
+        else:
+            pass
+
+    def test_prediction_elecpriceakkudoktor(self, server_setup_for_class, is_system_test):
+        """Test electricity price prediction by ElecPriceImport."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        # Reset config
+        with FILE_TESTDATA_EOSSERVER_CONFIG_1.open("r", encoding="utf-8", newline=None) as fd:
+            config = json.load(fd)
+        config["elecprice"]["provider"] = "ElecPriceAkkudoktor"
+        result = requests.put(f"{server}/v1/config", json=config)
+        assert result.status_code == HTTPStatus.OK
+
+        # Assure prediction is enabled
+        result = requests.get(f"{server}/v1/prediction/providers?enabled=true")
+        assert result.status_code == HTTPStatus.OK
+        providers = result.json()
+        assert "ElecPriceAkkudoktor" in providers
+
+        if is_system_test:
+            result = requests.post(f"{server}/v1/prediction/update/ElecPriceAkkudoktor")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=elecprice_marketprice_wh")
+            assert result.status_code == HTTPStatus.OK
+
+            data = result.json()
+            assert len(data["data"]) > 24
+
+        else:
+            pass
+
+    def test_prediction_loadakkudoktor(self, server_setup_for_class, is_system_test):
+        """Test load prediction by LoadAkkudoktor."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        result = requests.put(f"{server}/v1/config/load/provider", json="LoadAkkudoktor")
+        assert result.status_code == HTTPStatus.OK
+
+        # Assure prediction is enabled
+        result = requests.get(f"{server}/v1/prediction/providers?enabled=true")
+        assert result.status_code == HTTPStatus.OK
+        providers = result.json()
+        assert "LoadAkkudoktor" in providers
+
+        if is_system_test:
+            result = requests.post(f"{server}/v1/prediction/update/LoadAkkudoktor")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=load_mean")
+            assert result.status_code == HTTPStatus.OK
+
+            data = result.json()
+            assert len(data["data"]) > 24
+
+        else:
+            pass
+
+    def test_admin_cache(self, server_setup_for_class, is_system_test):
+        """Test whether cache is reconstructed from cached files."""
+        server = server_setup_for_class["server"]
+        eos_dir = server_setup_for_class["eos_dir"]
+
+        result = requests.get(f"{server}/v1/admin/cache")
+        assert result.status_code == HTTPStatus.OK
+        cache = result.json()
+
+        if is_system_test:
+            # There should be some cache data
+            assert cache != {}
+
+            # Save cache
+            result = requests.post(f"{server}/v1/admin/cache/save")
+            assert result.status_code == HTTPStatus.OK
+            cache_saved = result.json()
+            assert cache_saved == cache
+
+            # Clear cache - should clear nothing as all cache files expire in the future
+            result = requests.post(f"{server}/v1/admin/cache/clear")
+            assert result.status_code == HTTPStatus.OK
+            cache_cleared = result.json()
+            assert cache_cleared == cache
+
+            # Force clear cache
+            result = requests.post(f"{server}/v1/admin/cache/clear?clear_all=true")
+            assert result.status_code == HTTPStatus.OK
+            cache_cleared = result.json()
+            assert cache_cleared == {}
+
+            # Try to load already deleted cache entries
+            result = requests.post(f"{server}/v1/admin/cache/load")
+            assert result.status_code == HTTPStatus.OK
+            cache_loaded = result.json()
+            assert cache_loaded == {}
+
+            # Cache should still be empty
+            result = requests.get(f"{server}/v1/admin/cache")
+            assert result.status_code == HTTPStatus.OK
+            cache = result.json()
+            assert cache == {}
+
+
+class TestServerStartStop:
+    def test_server_start_eosdash(self, tmpdir):
+        """Test the EOSdash server startup from EOS."""
+        # Do not use any fixture as this will make pytest the owner of the EOSdash port.
+        if os.name == "nt":
+            host = "localhost"
+            # Windows does not provide SIGKILL
+            sigkill = signal.SIGTERM
+        else:
+            host = "0.0.0.0"
+            sigkill = signal.SIGKILL
+        port = 8503
+        eosdash_port = 8504
+        timeout = 120
+
+        server = f"http://{host}:{port}"
+        eosdash_server = f"http://{host}:{eosdash_port}"
+        eos_dir = str(tmpdir)
+
+        # Cleanup any EOSdash process left.
+        try:
+            result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+            if result.status_code == HTTPStatus.OK:
+                pid = result.json()["pid"]
+                os.kill(pid, sigkill)
+                time.sleep(1)
+                result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+                assert result.status_code != HTTPStatus.OK
+        except:
+            pass
+
+        # Wait for EOSdash port to be freed
+        process_info: list[dict] = []
+        for retries in range(int(timeout / 3)):
+            process_info = []
+            pids: list[int] = []
+            for conn in psutil.net_connections(kind="inet"):
+                if conn.laddr.port == eosdash_port:
+                    if conn.pid not in pids:
+                        # Get fresh process info
+                        process = psutil.Process(conn.pid)
+                        pids.append(conn.pid)
+                        process_info.append(process.as_dict(attrs=["pid", "cmdline"]))
+            if len(process_info) == 0:
+                break
+            time.sleep(3)
+        assert len(process_info) == 0
+
+        # Import after test setup to prevent creation of config file before test
+        from akkudoktoreos.server.eos import start_eosdash
+
+        process = start_eosdash(
+            host=host,
+            port=eosdash_port,
+            eos_host=host,
+            eos_port=port,
+            log_level="debug",
+            access_log=False,
+            reload=False,
+            eos_dir=eos_dir,
+            eos_config_dir=eos_dir,
+        )
+
+        # Assure EOSdash is up
+        startup = False
+        error = ""
+        for retries in range(int(timeout / 3)):
+            try:
+                result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+                if result.status_code == HTTPStatus.OK:
+                    startup = True
+                    break
+                error = f"{result.status_code}, {str(result.content)}"
+            except Exception as ex:
+                error = str(ex)
+            time.sleep(3)
+
+        assert startup, f"Connection to {eosdash_server}/eosdash/health failed: {error}"
+        assert result.json()["status"] == "alive"
+
+        # Shutdown eosdash
+        try:
+            result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+            if result.status_code == HTTPStatus.OK:
+                pid = result.json()["pid"]
+                os.kill(pid, signal.SIGTERM)
+                time.sleep(1)
+                result = requests.get(f"{eosdash_server}/eosdash/health", timeout=2)
+                assert result.status_code != HTTPStatus.OK
+        except:
+            pass
+
+    @pytest.mark.skipif(os.name == "nt", reason="Server restart not supported on Windows")
+    def test_server_restart(self, server_setup_for_function, is_system_test):
+        """Test server restart."""
+        server = server_setup_for_function["server"]
+        eos_dir = server_setup_for_function["eos_dir"]
+        timeout = server_setup_for_function["timeout"]
+
+        result = requests.get(f"{server}/v1/config")
+        assert result.status_code == HTTPStatus.OK
+
+        # Get testing config
+        config_json = result.json()
+        config_folder_path = Path(config_json["general"]["config_folder_path"])
+        config_file_path = Path(config_json["general"]["config_file_path"])
+        data_folder_path = Path(config_json["general"]["data_folder_path"])
+        data_ouput_path = Path(config_json["general"]["data_output_path"])
+        cache_file_path = data_folder_path.joinpath(config_json["cache"]["subpath"]).joinpath(
+            "cachefilestore.json"
+        )
+        # Assure we are working in test environment
+        assert str(config_folder_path).startswith(eos_dir)
+        assert str(config_file_path).startswith(eos_dir)
+        assert str(data_folder_path).startswith(eos_dir)
+        assert str(data_ouput_path).startswith(eos_dir)
+
+        if is_system_test:
+            # Prepare cache entry and get cached data
+            result = requests.put(f"{server}/v1/config/weather/provider", json="BrightSky")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.post(f"{server}/v1/prediction/update/BrightSky")
+            assert result.status_code == HTTPStatus.OK
+
+            result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air")
+            assert result.status_code == HTTPStatus.OK
+            data = result.json()
+            assert data["data"] != {}
+
+            result = requests.put(f"{server}/v1/config/file")
+            assert result.status_code == HTTPStatus.OK
+
+        # Save cache
+        result = requests.post(f"{server}/v1/admin/cache/save")
+        assert result.status_code == HTTPStatus.OK
+        cache = result.json()
+
+        assert cache_file_path.exists()
+
+        result = requests.get(f"{server}/v1/admin/cache")
+        assert result.status_code == HTTPStatus.OK
+        cache = result.json()
+
+        result = requests.get(f"{server}/v1/health")
+        assert result.status_code == HTTPStatus.OK
+        pid = result.json()["pid"]
+
+        result = requests.post(f"{server}/v1/admin/server/restart")
+        assert result.status_code == HTTPStatus.OK
+        assert "Restarting EOS.." in result.json()["message"]
+        new_pid = result.json()["pid"]
+
+        # Wait for server to shut down
+        for retries in range(10):
+            try:
+                result = requests.get(f"{server}/v1/health", timeout=2)
+                if result.status_code == HTTPStatus.OK:
+                    pid = result.json()["pid"]
+                    if pid == new_pid:
+                        # Already started
+                        break
+                else:
+                    break
+            except Exception as ex:
+                break
+            time.sleep(3)
+
+        # Assure EOS is up again
+        startup = False
+        error = ""
+        for retries in range(int(timeout / 3)):
+            try:
+                result = requests.get(f"{server}/v1/health", timeout=2)
+                if result.status_code == HTTPStatus.OK:
+                    startup = True
+                    break
+                error = f"{result.status_code}, {str(result.content)}"
+            except Exception as ex:
+                error = str(ex)
+            time.sleep(3)
+
+        assert startup, f"Connection to {server}/v1/health failed: {error}"
+        assert result.json()["status"] == "alive"
+        pid = result.json()["pid"]
+        assert pid == new_pid
+
+        result = requests.get(f"{server}/v1/admin/cache")
+        assert result.status_code == HTTPStatus.OK
+        new_cache = result.json()
+
+        assert cache.items() <= new_cache.items()
+
+        if is_system_test:
+            result = requests.get(f"{server}/v1/config")
+            assert result.status_code == HTTPStatus.OK
+            assert result.json()["weather"]["provider"] == "BrightSky"
+
+            # Wait for initialisation task to have finished
+            time.sleep(5)
+
+            result = requests.get(f"{server}/v1/prediction/series?key=weather_temp_air")
+            assert result.status_code == HTTPStatus.OK
+            assert result.json() == data
+
+        # Shutdown the newly created server
+        result = requests.post(f"{server}/v1/admin/server/shutdown")
+        assert result.status_code == HTTPStatus.OK
+        assert "Stopping EOS.." in result.json()["message"]
+        new_pid = result.json()["pid"]

tests/test_weatherbrightsky.py

@@ -5,9 +5,9 @@ from unittest.mock import Mock, patch
 import pandas as pd
 import pytest
 
+from akkudoktoreos.core.cache import CacheFileStore
 from akkudoktoreos.core.ems import get_ems
 from akkudoktoreos.prediction.weatherbrightsky import WeatherBrightSky
-from akkudoktoreos.utils.cacheutil import CacheFileStore
 from akkudoktoreos.utils.datetimeutil import to_datetime
 
 DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata")
@@ -20,15 +20,15 @@ FILE_TESTDATA_WEATHERBRIGHTSKY_2_JSON = DIR_TESTDATA.joinpath("weatherforecast_b
 def provider(monkeypatch):
     """Fixture to create a WeatherProvider instance."""
     monkeypatch.setenv("EOS_WEATHER__WEATHER_PROVIDER", "BrightSky")
-    monkeypatch.setenv("EOS_PREDICTION__LATITUDE", "50.0")
-    monkeypatch.setenv("EOS_PREDICTION__LONGITUDE", "10.0")
+    monkeypatch.setenv("EOS_GENERAL__LATITUDE", "50.0")
+    monkeypatch.setenv("EOS_GENERAL__LONGITUDE", "10.0")
     return WeatherBrightSky()
 
 
 @pytest.fixture
 def sample_brightsky_1_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_WEATHERBRIGHTSKY_1_JSON, "r") as f_res:
+    with FILE_TESTDATA_WEATHERBRIGHTSKY_1_JSON.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = json.load(f_res)
     return input_data
 
@@ -36,7 +36,7 @@ def sample_brightsky_1_json():
 @pytest.fixture
 def sample_brightsky_2_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_WEATHERBRIGHTSKY_2_JSON, "r") as f_res:
+    with FILE_TESTDATA_WEATHERBRIGHTSKY_2_JSON.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = json.load(f_res)
     return input_data
 
@@ -173,15 +173,18 @@ def test_update_data(mock_get, provider, sample_brightsky_1_json, cache_store):
 # ------------------------------------------------
 
 
-@pytest.mark.skip(reason="For development only")
-def test_brightsky_development_forecast_data(provider):
+def test_brightsky_development_forecast_data(provider, config_eos, is_system_test):
     """Fetch data from real BrightSky server."""
+    if not is_system_test:
+        return
+
     # Preset, as this is usually done by update_data()
-    provider.start_datetime = to_datetime("2024-10-26 00:00:00")
-    provider.latitude = 50.0
-    provider.longitude = 10.0
+    ems_eos = get_ems()
+    ems_eos.set_start_datetime(to_datetime("2024-10-26 00:00:00", in_timezone="Europe/Berlin"))
+    config_eos.general.latitude = 50.0
+    config_eos.general.longitude = 10.0
 
     brightsky_data = provider._request_forecast()
 
-    with open(FILE_TESTDATA_WEATHERBRIGHTSKY_1_JSON, "w") as f_out:
+    with FILE_TESTDATA_WEATHERBRIGHTSKY_1_JSON.open("w", encoding="utf-8", newline="\n") as f_out:
         json.dump(brightsky_data, f_out, indent=4)

tests/test_weatherclearoutside.py

@@ -9,9 +9,9 @@ import pvlib
 import pytest
 from bs4 import BeautifulSoup
 
+from akkudoktoreos.core.cache import CacheFileStore
 from akkudoktoreos.core.ems import get_ems
 from akkudoktoreos.prediction.weatherclearoutside import WeatherClearOutside
-from akkudoktoreos.utils.cacheutil import CacheFileStore
 from akkudoktoreos.utils.datetimeutil import compare_datetimes, to_datetime
 
 DIR_TESTDATA = Path(__file__).absolute().parent.joinpath("testdata")
@@ -39,7 +39,9 @@ def provider(config_eos):
 @pytest.fixture
 def sample_clearout_1_html():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_HTML, "r") as f_res:
+    with FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_HTML.open(
+        "r", encoding="utf-8", newline=None
+    ) as f_res:
         input_data = f_res.read()
     return input_data
 
@@ -47,7 +49,7 @@ def sample_clearout_1_html():
 @pytest.fixture
 def sample_clearout_1_data():
     """Fixture that returns sample forecast data."""
-    with open(FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_DATA, "r", encoding="utf8") as f_in:
+    with FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_DATA.open("r", encoding="utf-8", newline=None) as f_in:
         json_str = f_in.read()
         data = WeatherClearOutside.from_json(json_str)
     return data
@@ -220,7 +222,9 @@ def test_development_forecast_data(mock_get, provider, sample_clearout_1_html):
     # Fill the instance
     provider.update_data(force_enable=True)
 
-    with open(FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_DATA, "w", encoding="utf8") as f_out:
+    with FILE_TESTDATA_WEATHERCLEAROUTSIDE_1_DATA.open(
+        "w", encoding="utf-8", newline="\n"
+    ) as f_out:
         f_out.write(provider.to_json())
 
 

tests/test_weatherimport.py

@@ -33,7 +33,7 @@ def provider(sample_import_1_json, config_eos):
 @pytest.fixture
 def sample_import_1_json():
     """Fixture that returns sample forecast data report."""
-    with open(FILE_TESTDATA_WEATHERIMPORT_1_JSON, "r") as f_res:
+    with FILE_TESTDATA_WEATHERIMPORT_1_JSON.open("r", encoding="utf-8", newline=None) as f_res:
         input_data = json.load(f_res)
     return input_data
 

tests/testdata/eosserver_config_1.json

@@ -0,0 +1,86 @@
+{
+  "elecprice": {
+    "charges_kwh": 0.21,
+    "provider": "ElecPriceImport"
+  },
+  "general": {
+    "latitude": 52.5,
+    "longitude": 13.4
+  },
+  "prediction": {
+    "historic_hours": 48,
+    "hours": 48
+  },
+  "load": {
+    "provider": "LoadImport",
+    "provider_settings": {
+      "loadakkudoktor_year_energy": 20000
+    }
+  },
+  "optimization": {
+    "hours": 48
+  },
+  "pvforecast": {
+    "planes": [
+      {
+        "peakpower": 5.0,
+        "surface_azimuth": -10,
+        "surface_tilt": 7,
+        "userhorizon": [
+          20,
+          27,
+          22,
+          20
+        ],
+        "inverter_paco": 10000
+      },
+      {
+        "peakpower": 4.8,
+        "surface_azimuth": -90,
+        "surface_tilt": 7,
+        "userhorizon": [
+          30,
+          30,
+          30,
+          50
+        ],
+        "inverter_paco": 10000
+      },
+      {
+        "peakpower": 1.4,
+        "surface_azimuth": -40,
+        "surface_tilt": 60,
+        "userhorizon": [
+          60,
+          30,
+          0,
+          30
+        ],
+        "inverter_paco": 2000
+      },
+      {
+        "peakpower": 1.6,
+        "surface_azimuth": 5,
+        "surface_tilt": 45,
+        "userhorizon": [
+          45,
+          25,
+          30,
+          60
+        ],
+        "inverter_paco": 1400
+      }
+    ],
+    "provider": "PVForecastImport"
+  },
+  "server": {
+    "startup_eosdash": true,
+    "host": "0.0.0.0",
+    "port": 8503,
+    "eosdash_host": "0.0.0.0",
+    "eosdash_port": 8504
+  },
+  "weather": {
+    "provider": "WeatherImport"
+  }
+}