feat: improve config backup and update and revert (#737)
Some checks failed
docker-build / platform-excludes (push) Has been cancelled
docker-build / build (push) Has been cancelled
docker-build / merge (push) Has been cancelled
pre-commit / pre-commit (push) Has been cancelled
Run Pytest on Pull Request / test (push) Has been cancelled

Improve the backup of the EOS configuration on configuration migration
from another version. Backup files now get a backup id based on date
and time.

Add the configuration backup listing and the revert to the backup to
the EOS api.

Add revert to backup to the EOSdash admin tab.

Improve documentation about install, update and revert of EOS versions.

Add EOS execution profiling to make commands and to test description in
the development guideline.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
Bobby Noelte
2025-11-03 17:40:25 +01:00
committed by GitHub
parent 3432116845
commit 94c4ee2951
14 changed files with 707 additions and 170 deletions

View File

@@ -1,5 +1,5 @@
# Define the targets # Define the targets
.PHONY: help venv pip install dist test test-full test-system test-ci docker-run docker-build docs read-docs clean format gitlint mypy run run-dev run-dash run-dash-dev bumps .PHONY: help venv pip install dist test test-full test-system test-ci test-profile docker-run docker-build docs read-docs clean format gitlint mypy run run-dev run-dash run-dash-dev bumps
# Default target # Default target
all: help all: help
@@ -28,6 +28,7 @@ help:
@echo " test-full - Run tests with full optimization." @echo " test-full - Run tests with full optimization."
@echo " test-system - Run tests with system tests enabled." @echo " test-system - Run tests with system tests enabled."
@echo " test-ci - Run tests as CI does. No user config file allowed." @echo " test-ci - Run tests as CI does. No user config file allowed."
@echo " test-profile - Run single test optimization with profiling."
@echo " dist - Create distribution (in dist/)." @echo " dist - Create distribution (in dist/)."
@echo " clean - Remove generated documentation, distribution and virtual environment." @echo " clean - Remove generated documentation, distribution and virtual environment."
@echo " bump - Bump version to next release version." @echo " bump - Bump version to next release version."
@@ -136,6 +137,11 @@ test-full:
@echo "Running all tests..." @echo "Running all tests..."
.venv/bin/pytest --full-run .venv/bin/pytest --full-run
# Target to run tests including the single test optimization with profiling.
test-profile:
@echo "Running single test optimization with profiling..."
.venv/bin/python tests/single_test_optimization.py --profile
# Target to format code. # Target to format code.
format: format:
.venv/bin/pre-commit run --all-files .venv/bin/pre-commit run --all-files

View File

@@ -363,6 +363,25 @@ Returns:
--- ---
## GET /v1/config/backup
**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_backup_get_v1_config_backup_get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_backup_get_v1_config_backup_get)
Fastapi Config Backup Get
```
Get the EOS configuration backup identifiers and backup metadata.
Returns:
dict[str, dict[str, Any]]: Mapping of backup identifiers to metadata.
```
**Responses**:
- **200**: Successful Response
---
## PUT /v1/config/file ## PUT /v1/config/file
**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_file_put_v1_config_file_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_file_put_v1_config_file_put) **Links**: [local](http://localhost:8503/docs#/default/fastapi_config_file_put_v1_config_file_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_file_put_v1_config_file_put)
@@ -401,6 +420,31 @@ Returns:
--- ---
## PUT /v1/config/revert
**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_revert_put_v1_config_revert_put), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_revert_put_v1_config_revert_put)
Fastapi Config Revert Put
```
Revert the configuration to a EOS configuration backup.
Returns:
configuration (ConfigEOS): The current configuration after revert.
```
**Parameters**:
- `backup_id` (query, required): EOS configuration backup ID.
**Responses**:
- **200**: Successful Response
- **422**: Validation Error
---
## GET /v1/config/{path} ## GET /v1/config/{path}
**Links**: [local](http://localhost:8503/docs#/default/fastapi_config_get_key_v1_config__path__get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_get_key_v1_config__path__get) **Links**: [local](http://localhost:8503/docs#/default/fastapi_config_get_key_v1_config__path__get), [eos](https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/Akkudoktor-EOS/EOS/refs/heads/main/openapi.json#/default/fastapi_config_get_key_v1_config__path__get)

View File

@@ -99,7 +99,7 @@ html_theme_options = {
"logo_only": False, "logo_only": False,
"titles_only": True, "titles_only": True,
} }
html_css_files = ["eos.css"] html_css_files = ["eos.css"] # Make body size wider
# -- Options for autodoc ------------------------------------------------- # -- Options for autodoc -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html # https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html

View File

@@ -322,10 +322,7 @@ interfere with the EOS server trying to start EOSdash.
docker rm -f akkudoktoreos docker rm -f akkudoktoreos
``` ```
For detailed Docker instructions, refer to For detailed Docker instructions, refer to [Installation Guideline](install-page)
**[Method 3 & 4: Installation with Docker](install.md#method-3-installation-with-docker-dockerhub)**
and
**[Method 4: Docker Compose](install.md#method-4-installation-with-docker-docker-compose)**.
### Step 4 - Create the changes ### Step 4 - Create the changes
@@ -421,6 +418,30 @@ resources:
make test-system make test-system
``` ```
To do profiling use:
```{eval-rst}
.. tabs::
.. tab:: Windows
.. code-block:: powershell
python tests/single_test_optimization.py --profile
.. tab:: Linux
.. code-block:: bash
python tests/single_test_optimization.py --profile
.. tab:: Linux Make
.. code-block:: bash
make test-profile
```
#### Step 4.5 - Commit the changes #### Step 4.5 - Commit the changes
Add the changed and new files to the commit. Add the changed and new files to the commit.

View File

@@ -3,28 +3,42 @@
# Installation Guide # Installation Guide
This guide provides four different methods to install AkkudoktorEOS. Choose the method that best This guide provides different methods to install AkkudoktorEOS:
suits your needs.
- Installation from Source (GitHub)
- Installation from Release Package (GitHub)
- Installation with Docker (DockerHub)
- Installation with Docker (docker-compose)
Choose the method that best suits your needs.
:::{admonition} Tip
:class: Note
If you need to update instead, see the [Update Guideline](update-page). For reverting to a previous
release see the [Revert Guideline](revert-page).
:::
## Installation Prerequisites ## Installation Prerequisites
Before installing, ensure you have the following: Before installing, ensure you have the following:
- **For Source/Release Installation:** ### For Source / Release Installation
- Python 3.10 or higher
- pip (Python package manager)
- Git (for source installation)
- Tar/Zip (for release package installation)
- **For Docker Installation:** - Python 3.10 or higher
- Docker Engine 20.10 or higher - pip
- Docker Compose (optional, but recommended) - Git (only for source)
- Tar/Zip (for release package)
## Method 1: Installation from Source (GitHub) ### For Docker Installation
This method is recommended for developers or users who want the latest features. - Docker Engine 20.10 or higher
- Docker Compose (optional, recommended)
### M1-Step 1: Clone the Repository ## Installation from Source (GitHub) (M1)
Recommended for developers or users wanting the latest updates.
### 1) Clone the Repository (M1)
```{eval-rst} ```{eval-rst}
.. tabs:: .. tabs::
@@ -44,7 +58,7 @@ This method is recommended for developers or users who want the latest features.
cd EOS cd EOS
``` ```
### M1-Step 2: Create a Virtual Environment and install dependencies ### 2) Create a Virtual Environment and install dependencies (M1)
```{eval-rst} ```{eval-rst}
.. tabs:: .. tabs::
@@ -67,7 +81,7 @@ This method is recommended for developers or users who want the latest features.
``` ```
### M1-Step 3: Run EOS ### 3) Run EOS (M1)
```{eval-rst} ```{eval-rst}
.. tabs:: .. tabs::
@@ -86,8 +100,10 @@ This method is recommended for developers or users who want the latest features.
``` ```
EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash EOS is now available at:
should be available at [http://localhost:8504](http://localhost:8504).
- API: [http://localhost:8503/docs](http://localhost:8503/docs)
- EOSdash: [http://localhost:8504](http://localhost:8504)
If you want to make EOS and EOSdash accessible from outside of your machine or container at this If you want to make EOS and EOSdash accessible from outside of your machine or container at this
stage of the installation provide appropriate IP addresses on startup. stage of the installation provide appropriate IP addresses on startup.
@@ -111,43 +127,20 @@ stage of the installation provide appropriate IP addresses on startup.
``` ```
<!-- pyml enable line-length --> <!-- pyml enable line-length -->
### M1-Step 4: Configure EOS ### 4) Configure EOS (M1)
Use [EOSdash](http://localhost:8504) to configure EOS. Use EOSdash at [http://localhost:8504](http://localhost:8504) to configure EOS.
### Updating from Source ## Installation from Release Package (GitHub) (M2)
To update to the latest version:
```{eval-rst}
.. tabs::
.. tab:: Windows
.. code-block:: powershell
git pull origin main
.venv\Scripts\pip install -r requirements.txt --upgrade
.. tab:: Linux
.. code-block:: bash
git pull origin main
.venv/bin/pip install -r requirements.txt --upgrade
```
## Method 2: Installation from Release Package (GitHub)
This method is recommended for users who want a stable, tested version. This method is recommended for users who want a stable, tested version.
### M2-Step 1: Download the Latest Release ### 1) Download the Latest Release (M2)
Visit the [Releases page](https://github.com/Akkudoktor-EOS/EOS/releases) and download the latest Visit the [Releases page](https://github.com/Akkudoktor-EOS/EOS/tags) and download the latest
release package (e.g., `akkudoktoreos-v0.1.0.tar.gz` or `akkudoktoreos-v0.1.0.zip`). release package (e.g., `akkudoktoreos-v0.1.0.tar.gz` or `akkudoktoreos-v0.1.0.zip`).
### M2-Step 2: Extract the Package ### 2) Extract the Package (M2)
```bash ```bash
tar -xzf akkudoktoreos-v0.1.0.tar.gz # For .tar.gz tar -xzf akkudoktoreos-v0.1.0.tar.gz # For .tar.gz
@@ -157,15 +150,22 @@ unzip akkudoktoreos-v0.1.0.zip # For .zip
cd akkudoktoreos-v0.1.0 cd akkudoktoreos-v0.1.0
``` ```
### Follow Step 2, 3 and 4 of Method 1: Installation from source ### 3) Create a virtual environment and run and configure EOS (M2)
Installation from release package now needs the exact same steps 2, 3, 4 of method 1. Follow Step 2), 3) and 4) of method M1. Start at
`2) Create a Virtual Environment and install dependencies`
## Method 3: Installation with Docker (DockerHub) ### 4) Update the source code (M2)
To extract a new release to a new directory just proceed with method M2 step 1) for the new release.
You may remove the old release directory afterwards.
## Installation with Docker (DockerHub) (M3)
This method is recommended for easy deployment and containerized environments. This method is recommended for easy deployment and containerized environments.
### M3-Step 1: Pull the Docker Image ### 1) Pull the Docker Image (M3)
```bash ```bash
docker pull akkudoktor/eos:latest docker pull akkudoktor/eos:latest
@@ -174,10 +174,10 @@ docker pull akkudoktor/eos:latest
For a specific version: For a specific version:
```bash ```bash
docker pull akkudoktor/eos:v0.1.0 docker pull akkudoktor/eos:v<version>
``` ```
### M3-Step 2: Run the Container ### 2) Run the Container (M3)
**Basic run:** **Basic run:**
@@ -199,7 +199,7 @@ docker run -d \
akkudoktor/eos:latest akkudoktor/eos:latest
``` ```
### M3-Step 3: Verify the Container is Running ### 3) Verify the Container is Running (M3)
```bash ```bash
docker ps docker ps
@@ -209,11 +209,50 @@ docker logs akkudoktoreos
EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash
should be available at [http://localhost:8504](http://localhost:8504). should be available at [http://localhost:8504](http://localhost:8504).
### M3-Step 4: Configure EOS ### 4) Configure EOS (M3)
Use [EOSdash](http://localhost:8504) to configure EOS. Use EOSdash at [http://localhost:8504](http://localhost:8504) to configure EOS.
### Docker Management Commands ## Installation with Docker (docker-compose) (M4)
### 1) Get the akkudoktoreos source code (M4)
You may use either method M1 or method M2 to get the source code.
### 2) Build and run the container (M4)
```{eval-rst}
.. tabs::
.. tab:: Windows
.. code-block:: powershell
docker compose up --build
.. tab:: Linux
.. code-block:: bash
docker compose up --build
```
### 3) Verify the Container is Running (M4)
```bash
docker ps
docker logs akkudoktoreos
```
EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash
should be available at [http://localhost:8504](http://localhost:8504).
### 4) Configure EOS
Use EOSdash at [http://localhost:8504](http://localhost:8504) to configure EOS.
## Helpful Docker Commands
**View logs:** **View logs:**
@@ -247,42 +286,3 @@ docker stop akkudoktoreos
docker rm akkudoktoreos docker rm akkudoktoreos
# Then run the container again with the run command # Then run the container again with the run command
``` ```
## Method 4: Installation with Docker (docker-compose)
### M4-Step 1: Get the akkudoktoreos source code
You may use either method 1 or method 2 to get the source code.
### M4-Step 2: Build and run the container
```{eval-rst}
.. tabs::
.. tab:: Windows
.. code-block:: powershell
docker compose up --build
.. tab:: Linux
.. code-block:: bash
docker compose up --build
```
### M4-Step 3: Verify the Container is Running
```bash
docker ps
docker logs akkudoktoreos
```
EOS should now be accessible at [http://localhost:8503/docs](http://localhost:8503/docs) and EOSdash
should be available at [http://localhost:8504](http://localhost:8504).
### M4-Step 4: Configure EOS
Use [EOSdash](http://localhost:8504) to configure EOS.

155
docs/develop/revert.md Normal file
View File

@@ -0,0 +1,155 @@
% SPDX-License-Identifier: Apache-2.0
(revert-page)=
# Revert Guide
This guide explains how to **revert AkkudoktorEOS to a previous version**.
The exact methods and steps differ depending on how EOS was installed:
- M1/M2: Reverting when Installed from Source or Release Package
- M3/M4: Reverting when Installed via Docker
:::{admonition} Important
:class: warning
Before reverting, ensure you have a backup of your `EOS.config.json`.
EOS also maintains internal configuration backups that can be restored after a downgrade.
:::
:::{admonition} Tip
:class: Note
If you need to update instead, see the [Update Guideline](update-page).
:::
## Revert to a Previous Version of EOS
You can revert to a previous version using the same installation method you originally selected.
See: [Installation Guideline](install-page)
## Reverting when Installed from Source or Release Package (M1/M2)
### 1) Locate the target version (M2)
Go to the GitHub Releases page:
> <https://github.com/Akkudoktor-EOS/EOS/tags>
### 2) Download or check out that version (M1/M2)
#### Git (source) (M1)
```bash
git fetch
git checkout v<version>
````
Example:
```bash
git checkout v0.1.0
```
Then reinstall dependencies:
```bash
.venv/bin/pip install -r requirements.txt --upgrade
```
#### Release package (M2)
Download and extract the desired ZIP or TAR release.
Refer to **Method 2** in the [Installation Guideline](install-page).
### 3) Restart EOS (M1/M2)
```bash
.venv/bin/python -m akkudoktoreos.server.eos
```
### 4) Restore configuration (optional) (M1/M2)
If your configuration changed since the downgrade, you may restore a previous backup:
- via **EOSdash**
Admin → configuration → Revert to backup
or
Admin → configuration → Import from file
- via **REST**
```bash
curl -X PUT "http://<host>:8503/v1/config/revert?backup_id=<backup>"
```
## Reverting when Installed via Docker (M3/M4)
### 1) Pull the desired image version (M3/M4)
```bash
docker pull akkudoktor/eos:v<version>
```
Example:
```bash
docker pull akkudoktor/eos:v0.1.0
```
### 2) Stop and remove the current container (M3/M4)
```bash
docker stop akkudoktoreos
docker rm akkudoktoreos
```
### 3) Start a container with the selected version (M3/M4)
Start EOS as usual, using your existing `docker run` or `docker compose` setup
(see Method 3 or Method 4 in the [Installation Guideline](install-page)).
### 4) Restore configuration (optional) (M3/M4)
In many cases configuration will migrate automatically.
If needed, you may restore a configuration backup:
- via **EOSdash**
Admin → configuration → Revert to backup
or
Admin → configuration → Import from file
- via **REST**
```bash
curl -X PUT "http://<host>:8503/v1/config/revert?backup_id=<backup>"
```
## About Configuration Backups
EOS keeps configuration backup files next to your active `EOS.config.json`.
You can list and restore backups:
- via **EOSdash UI**
- via **REST API**
### List available backups
```bash
GET /v1/config/backups
```
### Restore backup
```bash
PUT /v1/config/revert?backup_id=<id>
```
:::{admonition} Important
:class: warning
If no backup file is available, create or copy a previously saved `EOS.config.json` before reverting.
:::

View File

@@ -27,6 +27,9 @@ develop/getting_started.md
:caption: How-To Guides :caption: How-To Guides
develop/CONTRIBUTING.md develop/CONTRIBUTING.md
develop/install.md
develop/update.md
develop/revert.md
``` ```
@@ -53,8 +56,6 @@ akkudoktoreos/api.rst
:maxdepth: 2 :maxdepth: 2
:caption: Development :caption: Development
develop/CONTRIBUTING.md
develop/install.md
develop/develop.md develop/develop.md
develop/release.md develop/release.md
develop/CHANGELOG.md develop/CHANGELOG.md

View File

@@ -213,6 +213,78 @@
} }
} }
}, },
"/v1/config/backup": {
"get": {
"tags": [
"config"
],
"summary": "Fastapi Config Backup Get",
"description": "Get the EOS configuration backup identifiers and backup metadata.\n\nReturns:\n dict[str, dict[str, Any]]: Mapping of backup identifiers to metadata.",
"operationId": "fastapi_config_backup_get_v1_config_backup_get",
"responses": {
"200": {
"description": "Successful Response",
"content": {
"application/json": {
"schema": {
"additionalProperties": {
"additionalProperties": true,
"type": "object"
},
"type": "object",
"title": "Response Fastapi Config Backup Get V1 Config Backup Get"
}
}
}
}
}
}
},
"/v1/config/revert": {
"put": {
"tags": [
"config"
],
"summary": "Fastapi Config Revert Put",
"description": "Revert the configuration to a EOS configuration backup.\n\nReturns:\n configuration (ConfigEOS): The current configuration after revert.",
"operationId": "fastapi_config_revert_put_v1_config_revert_put",
"parameters": [
{
"name": "backup_id",
"in": "query",
"required": true,
"schema": {
"type": "string",
"description": "EOS configuration backup ID.",
"title": "Backup Id"
},
"description": "EOS configuration backup ID."
}
],
"responses": {
"200": {
"description": "Successful Response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ConfigEOS"
}
}
}
},
"422": {
"description": "Validation Error",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/HTTPValidationError"
}
}
}
}
}
}
},
"/v1/config/file": { "/v1/config/file": {
"put": { "put": {
"tags": [ "tags": [

View File

@@ -9,6 +9,7 @@ Key features:
- Managing directory setups for the application - Managing directory setups for the application
""" """
import json
import os import os
import shutil import shutil
from pathlib import Path from pathlib import Path
@@ -21,7 +22,7 @@ from pydantic import Field, computed_field, field_validator
# settings # settings
from akkudoktoreos.config.configabc import SettingsBaseModel from akkudoktoreos.config.configabc import SettingsBaseModel
from akkudoktoreos.config.configmigrate import migrate_config_file from akkudoktoreos.config.configmigrate import migrate_config_data, migrate_config_file
from akkudoktoreos.core.cachesettings import CacheCommonSettings from akkudoktoreos.core.cachesettings import CacheCommonSettings
from akkudoktoreos.core.coreabc import SingletonMixin from akkudoktoreos.core.coreabc import SingletonMixin
from akkudoktoreos.core.decorators import classproperty from akkudoktoreos.core.decorators import classproperty
@@ -41,7 +42,7 @@ from akkudoktoreos.prediction.prediction import PredictionCommonSettings
from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings from akkudoktoreos.prediction.pvforecast import PVForecastCommonSettings
from akkudoktoreos.prediction.weather import WeatherCommonSettings from akkudoktoreos.prediction.weather import WeatherCommonSettings
from akkudoktoreos.server.server import ServerCommonSettings from akkudoktoreos.server.server import ServerCommonSettings
from akkudoktoreos.utils.datetimeutil import to_timezone from akkudoktoreos.utils.datetimeutil import to_datetime, to_timezone
from akkudoktoreos.utils.utils import UtilsCommonSettings from akkudoktoreos.utils.utils import UtilsCommonSettings
@@ -379,9 +380,9 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
# Apend file settings to sources # Apend file settings to sources
file_settings: Optional[pydantic_settings.JsonConfigSettingsSource] = None file_settings: Optional[pydantic_settings.JsonConfigSettingsSource] = None
try: try:
backup_file = config_file.with_suffix(".bak") backup_file = config_file.with_suffix(f".{to_datetime(as_string='YYYYMMDDHHmmss')}")
if migrate_config_file(config_file, backup_file): if migrate_config_file(config_file, backup_file):
# If correct version add it as settings source # If the config file does have the correct version add it as settings source
file_settings = pydantic_settings.JsonConfigSettingsSource( file_settings = pydantic_settings.JsonConfigSettingsSource(
settings_cls, json_file=config_file settings_cls, json_file=config_file
) )
@@ -478,6 +479,88 @@ class ConfigEOS(SingletonMixin, SettingsEOSDefaults):
""" """
self._setup() self._setup()
def revert_settings(self, backup_id: str) -> None:
"""Revert application settings to a stored backup.
This method restores configuration values from a backup file identified
by `backup_id`. The backup is expected to exist alongside the main
configuration file, using the main config file's path but with the given
suffix. Any settings previously applied will be overwritten.
Args:
backup_id (str): The suffix used to locate the backup configuration
file. Example: ``".bak"`` or ``".backup"``.
Returns:
None: The method does not return a value.
Raises:
ValueError: If the backup file cannot be found at the constructed path.
json.JSONDecodeError: If the backup file exists but contains invalid JSON.
TypeError: If the unpacked backup data fails to match the signature
required by ``self._setup()``.
OSError: If reading the backup file fails due to I/O issues.
"""
backup_file_path = self.general.config_file_path.with_suffix(f".{backup_id}")
if not backup_file_path.exists():
error_msg = f"Configuration backup `{backup_id}` not found."
logger.error(error_msg)
raise ValueError(error_msg)
with backup_file_path.open("r", encoding="utf-8") as f:
backup_data: dict[str, Any] = json.load(f)
backup_settings = migrate_config_data(backup_data)
self._setup(**backup_settings.model_dump(exclude_none=True, exclude_unset=True))
def list_backups(self) -> dict[str, dict[str, Any]]:
"""List available configuration backup files and extract metadata.
Backup files are identified by sharing the same stem as the main config
file but having a different suffix. Each backup file is assumed to contain
a JSON object.
The returned dictionary uses `backup_id` (suffix) as keys. The value for
each key is a dictionary including:
- ``storage_time``: The file modification timestamp in ISO-8601 format.
- ``version``: Version information found in the backup file
(defaults to ``"unknown"``).
Returns:
dict[str, dict[str, Any]]: Mapping of backup identifiers to metadata.
Raises:
OSError: If directory scanning or file reading fails.
json.JSONDecodeError: If a backup file cannot be parsed as JSON.
"""
result: dict[str, dict[str, Any]] = {}
base_path: Path = self.general.config_file_path
parent = base_path.parent
stem = base_path.stem
# Iterate files next to config file
for file in parent.iterdir():
if file.is_file() and file.stem == stem and file != base_path:
backup_id = file.suffix[1:]
# Read version from file
with file.open("r", encoding="utf-8") as f:
data: dict[str, Any] = json.load(f)
# Extract version safely
version = data.get("general", {}).get("version", "unknown")
# Read file modification time (OS-independent)
ts = file.stat().st_mtime
storage_time = to_datetime(ts, as_string=True)
result[backup_id] = {
"date_time": storage_time,
"version": version,
}
return result
def _create_initial_config_file(self) -> None: def _create_initial_config_file(self) -> None:
if self.general.config_file_path and not self.general.config_file_path.exists(): if self.general.config_file_path and not self.general.config_file_path.exists():
self.general.config_file_path.parent.mkdir(parents=True, exist_ok=True) self.general.config_file_path.parent.mkdir(parents=True, exist_ok=True)

View File

@@ -3,12 +3,16 @@
import json import json
import shutil import shutil
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, List, Set, Tuple, Union from typing import TYPE_CHECKING, Any, Callable, Dict, List, Set, Tuple, Union
from loguru import logger from loguru import logger
from akkudoktoreos.core.version import __version__ from akkudoktoreos.core.version import __version__
if TYPE_CHECKING:
# There are circular dependencies - only import here for type checking
from akkudoktoreos.config.config import SettingsEOSDefaults
# ----------------------------- # -----------------------------
# Global migration map constant # Global migration map constant
# ----------------------------- # -----------------------------
@@ -57,6 +61,79 @@ auto_count: int = 0
skipped_paths: List[str] = [] skipped_paths: List[str] = []
def migrate_config_data(config_data: Dict[str, Any]) -> "SettingsEOSDefaults":
"""Migrate configuration data to the current version settings.
Returns:
SettingsEOSDefaults: The migrated settings.
"""
global migrated_source_paths, mapped_count, auto_count, skipped_paths
# Reset globals at the start of each migration
migrated_source_paths = set()
mapped_count = 0
auto_count = 0
skipped_paths = []
from akkudoktoreos.config.config import SettingsEOSDefaults
new_config = SettingsEOSDefaults()
# 1) Apply explicit migration map
for old_path, mapping in MIGRATION_MAP.items():
new_path = None
transform = None
if mapping is None:
migrated_source_paths.add(old_path.strip("/"))
logger.debug(f"🗑️ Migration map: dropping '{old_path}'")
continue
if isinstance(mapping, tuple):
new_path, transform = mapping
else:
new_path = mapping
old_value = _get_json_nested_value(config_data, old_path)
if old_value is None:
migrated_source_paths.add(old_path.strip("/"))
mapped_count += 1
logger.debug(f"✅ Migrated mapped '{old_path}''None'")
continue
try:
if transform:
old_value = transform(old_value)
new_config.set_nested_value(new_path, old_value)
migrated_source_paths.add(old_path.strip("/"))
mapped_count += 1
logger.debug(f"✅ Migrated mapped '{old_path}''{new_path}' = {old_value!r}")
except Exception as e:
logger.opt(exception=True).warning(
f"Failed mapped migration '{old_path}' -> '{new_path}': {e}"
)
# 2) Automatic migration for remaining fields
auto_count += _migrate_matching_fields(
config_data, new_config, migrated_source_paths, skipped_paths
)
# 3) Ensure version
try:
new_config.set_nested_value("general/version", __version__)
except Exception as e:
logger.warning(f"Could not set version on new configuration model: {e}")
# 4) Log final migration summary
logger.info(
f"Migration summary: "
f"mapped fields: {mapped_count}, automatically migrated: {auto_count}, skipped: {len(skipped_paths)}"
)
if skipped_paths:
logger.debug(f"Skipped paths: {', '.join(skipped_paths)}")
logger.success(f"Configuration successfully migrated to version {__version__}.")
return new_config
def migrate_config_file(config_file: Path, backup_file: Path) -> bool: def migrate_config_file(config_file: Path, backup_file: Path) -> bool:
"""Migrate configuration file to the current version. """Migrate configuration file to the current version.
@@ -104,54 +181,10 @@ def migrate_config_file(config_file: Path, backup_file: Path) -> bool:
f"Failed to backup existing config (replace: {e_replace}; copy: {e_copy}). Continuing without backup." f"Failed to backup existing config (replace: {e_replace}; copy: {e_copy}). Continuing without backup."
) )
from akkudoktoreos.config.config import SettingsEOSDefaults # Migrate config data
new_config = migrate_config_data(config_data)
new_config = SettingsEOSDefaults() # Write migrated configuration
# 1) Apply explicit migration map
for old_path, mapping in MIGRATION_MAP.items():
new_path = None
transform = None
if mapping is None:
migrated_source_paths.add(old_path.strip("/"))
logger.debug(f"🗑️ Migration map: dropping '{old_path}'")
continue
if isinstance(mapping, tuple):
new_path, transform = mapping
else:
new_path = mapping
old_value = _get_json_nested_value(config_data, old_path)
if old_value is None:
migrated_source_paths.add(old_path.strip("/"))
mapped_count += 1
logger.debug(f"✅ Migrated mapped '{old_path}''None'")
continue
try:
if transform:
old_value = transform(old_value)
new_config.set_nested_value(new_path, old_value)
migrated_source_paths.add(old_path.strip("/"))
mapped_count += 1
logger.debug(f"✅ Migrated mapped '{old_path}''{new_path}' = {old_value!r}")
except Exception as e:
logger.opt(exception=True).warning(
f"Failed mapped migration '{old_path}' -> '{new_path}': {e}", exc_info=True
)
# 2) Automatic migration for remaining fields
auto_count += _migrate_matching_fields(
config_data, new_config, migrated_source_paths, skipped_paths
)
# 3) Ensure version
try:
new_config.set_nested_value("general/version", __version__)
except Exception as e:
logger.warning(f"Could not set version on new configuration model: {e}")
# 4) Write migrated configuration
try: try:
with config_file.open("w", encoding="utf-8", newline=None) as f_out: with config_file.open("w", encoding="utf-8", newline=None) as f_out:
json_str = new_config.model_dump_json(indent=4) json_str = new_config.model_dump_json(indent=4)
@@ -160,15 +193,6 @@ def migrate_config_file(config_file: Path, backup_file: Path) -> bool:
logger.error(f"Failed to write migrated configuration to '{config_file}': {e_write}") logger.error(f"Failed to write migrated configuration to '{config_file}': {e_write}")
return False return False
# 5) Log final migration summary
logger.info(
f"Migration summary for '{config_file}': "
f"mapped fields: {mapped_count}, automatically migrated: {auto_count}, skipped: {len(skipped_paths)}"
)
if skipped_paths:
logger.debug(f"Skipped paths: {', '.join(skipped_paths)}")
logger.success(f"Configuration successfully migrated to version {__version__}.")
return True return True
except Exception as e: except Exception as e:

View File

@@ -152,7 +152,11 @@ def AdminCache(
def AdminConfig( def AdminConfig(
eos_host: str, eos_port: Union[str, int], data: Optional[dict], config: Optional[dict[str, Any]] eos_host: str,
eos_port: Union[str, int],
data: Optional[dict],
config: Optional[dict[str, Any]],
config_backup: Optional[dict[str, dict[str, Any]]],
) -> tuple[str, Union[Card, list[Card]]]: ) -> tuple[str, Union[Card, list[Card]]]:
"""Creates a configuration management card with save-to-file functionality. """Creates a configuration management card with save-to-file functionality.
@@ -177,6 +181,8 @@ def AdminConfig(
config_file_path = get_nested_value(config, ["general", "config_file_path"]) config_file_path = get_nested_value(config, ["general", "config_file_path"])
except Exception as e: except Exception as e:
logger.debug(f"general.config_file_path: {e}") logger.debug(f"general.config_file_path: {e}")
# revert to backup
revert_to_backup_status = (None,)
# export config file # export config file
export_to_file_next_tag = to_datetime(as_string="YYYYMMDDHHmmss") export_to_file_next_tag = to_datetime(as_string="YYYYMMDDHHmmss")
export_to_file_status = (None,) export_to_file_status = (None,)
@@ -191,7 +197,7 @@ def AdminConfig(
result = requests.put(f"{server}/v1/config/file", timeout=10) result = requests.put(f"{server}/v1/config/file", timeout=10)
result.raise_for_status() result.raise_for_status()
config_file_path = result.json()["general"]["config_file_path"] config_file_path = result.json()["general"]["config_file_path"]
status = Success(f"Saved to '{config_file_path}' on '{eos_hostname}'") status = Success(f"Saved configuration to '{config_file_path}' on '{eos_hostname}'")
except requests.exceptions.HTTPError as e: except requests.exceptions.HTTPError as e:
detail = result.json()["detail"] detail = result.json()["detail"]
status = Error( status = Error(
@@ -199,6 +205,45 @@ def AdminConfig(
) )
except Exception as e: except Exception as e:
status = Error(f"Can not save actual config to file on '{eos_hostname}': {e}") status = Error(f"Can not save actual config to file on '{eos_hostname}': {e}")
elif data["action"] == "revert_to_backup":
# Revert configuration to backup file
metadata = data.get("backup_metadata", None)
if metadata and config_backup:
date_time = metadata.split(" ")[0]
backup_id = None
for bkup_id, bkup_meta in config_backup.items():
if bkup_meta.get("date_time") == date_time:
backup_id = bkup_id
break
if backup_id:
try:
result = requests.put(
f"{server}/v1/config/revert",
params={"backup_id": backup_id},
timeout=10,
)
result.raise_for_status()
config_file_path = result.json()["general"]["config_file_path"]
revert_to_backup_status = Success(
f"Reverted configuration to backup `{backup_id}` on '{eos_hostname}'"
)
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"]
revert_to_backup_status = Error(
f"Can not revert to backup `{backup_id}` on '{eos_hostname}': {e}, {detail}"
)
except Exception as e:
revert_to_backup_status = Error(
f"Can not revert to backup `{backup_id}` on '{eos_hostname}': {e}"
)
else:
revert_to_backup_status = Error(
f"Can not revert to backup `{backup_id}` on '{eos_hostname}': Invalid backup datetime {date_time}"
)
else:
revert_to_backup_status = Error(
f"Can not revert to backup configuration on '{eos_hostname}': No backup selected"
)
elif data["action"] == "export_to_file": elif data["action"] == "export_to_file":
# Export current configuration to file # Export current configuration to file
export_to_file_tag = data.get("export_to_file_tag", export_to_file_next_tag) export_to_file_tag = data.get("export_to_file_tag", export_to_file_next_tag)
@@ -257,6 +302,13 @@ def AdminConfig(
# Update for display, in case we added a new file before # Update for display, in case we added a new file before
import_from_file_names = [f.name for f in list(export_import_directory.glob("*.json"))] import_from_file_names = [f.name for f in list(export_import_directory.glob("*.json"))]
if config_backup is None:
revert_to_backup_metadata_list = ["Backup list not available"]
else:
revert_to_backup_metadata_list = [
f"{backup_meta['date_time']} {backup_meta['version']}"
for backup_id, backup_meta in config_backup.items()
]
return ( return (
category, category,
@@ -283,6 +335,33 @@ def AdminConfig(
P(f"Safe actual configuration to '{config_file_path}' on '{eos_hostname}'."), P(f"Safe actual configuration to '{config_file_path}' on '{eos_hostname}'."),
), ),
), ),
Card(
Details(
Summary(
Grid(
DivHStacked(
UkIcon(icon="play"),
AdminButton(
"Revert to backup",
hx_post="/eosdash/admin",
hx_target="#page-content",
hx_swap="innerHTML",
hx_vals='js:{ "category": "configuration", "action": "revert_to_backup", "backup_metadata": document.querySelector("[name=\'selected_backup_metadata\']").value }',
),
Select(
*Options(*revert_to_backup_metadata_list),
id="backup_metadata",
name="selected_backup_metadata", # Name of hidden input field with selected value
placeholder="Select backup",
),
),
revert_to_backup_status,
),
cls="list-none",
),
P(f"Revert configuration to backup on '{eosdash_hostname}'."),
),
),
Card( Card(
Details( Details(
Summary( Summary(
@@ -364,7 +443,20 @@ def Admin(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None)
result.raise_for_status() result.raise_for_status()
config = result.json() config = result.json()
except requests.exceptions.HTTPError as e: except requests.exceptions.HTTPError as e:
config = {} detail = result.json()["detail"]
warning_msg = f"Can not retrieve configuration from {server}: {e}, {detail}"
logger.warning(warning_msg)
return Error(warning_msg)
except Exception as e:
warning_msg = f"Can not retrieve configuration from {server}: {e}"
logger.warning(warning_msg)
return Error(warning_msg)
# Get current configuration backups from server
try:
result = requests.get(f"{server}/v1/config/backup", timeout=10)
result.raise_for_status()
config_backup = result.json()
except requests.exceptions.HTTPError as e:
detail = result.json()["detail"] detail = result.json()["detail"]
warning_msg = f"Can not retrieve configuration from {server}: {e}, {detail}" warning_msg = f"Can not retrieve configuration from {server}: {e}, {detail}"
logger.warning(warning_msg) logger.warning(warning_msg)
@@ -378,7 +470,7 @@ def Admin(eos_host: str, eos_port: Union[str, int], data: Optional[dict] = None)
last_category = "" last_category = ""
for category, admin in [ for category, admin in [
AdminCache(eos_host, eos_port, data, config), AdminCache(eos_host, eos_port, data, config),
AdminConfig(eos_host, eos_port, data, config), AdminConfig(eos_host, eos_port, data, config, config_backup),
]: ]:
if category != last_category: if category != last_category:
rows.append(H3(category)) rows.append(H3(category))

View File

@@ -639,6 +639,42 @@ def fastapi_config_reset_post() -> ConfigEOS:
return config_eos return config_eos
@app.get("/v1/config/backup", tags=["config"])
def fastapi_config_backup_get() -> dict[str, dict[str, Any]]:
"""Get the EOS configuration backup identifiers and backup metadata.
Returns:
dict[str, dict[str, Any]]: Mapping of backup identifiers to metadata.
"""
try:
result = config_eos.list_backups()
except Exception as e:
raise HTTPException(
status_code=404,
detail=f"Can not list configuration backups: {e}",
)
return result
@app.put("/v1/config/revert", tags=["config"])
def fastapi_config_revert_put(
backup_id: str = Query(..., description="EOS configuration backup ID."),
) -> ConfigEOS:
"""Revert the configuration to a EOS configuration backup.
Returns:
configuration (ConfigEOS): The current configuration after revert.
"""
try:
config_eos.revert_settings(backup_id)
except Exception as e:
raise HTTPException(
status_code=400,
detail=f"Error on reverting of configuration: {e}",
)
return config_eos
@app.put("/v1/config/file", tags=["config"]) @app.put("/v1/config/file", tags=["config"])
def fastapi_config_file_put() -> ConfigEOS: def fastapi_config_file_put() -> ConfigEOS:
"""Save the current configuration to the EOS configuration file. """Save the current configuration to the EOS configuration file.

View File

@@ -15,7 +15,7 @@ from loguru import logger
from akkudoktoreos.config.config import get_config from akkudoktoreos.config.config import get_config
from akkudoktoreos.core.ems import get_ems from akkudoktoreos.core.ems import get_ems
from akkudoktoreos.core.emsettings import EnergyManagementMode from akkudoktoreos.core.emsettings import EnergyManagementMode
from akkudoktoreos.optimization.genetic import ( from akkudoktoreos.optimization.genetic.geneticparams import (
GeneticOptimizationParameters, GeneticOptimizationParameters,
) )
from akkudoktoreos.prediction.prediction import get_prediction from akkudoktoreos.prediction.prediction import get_prediction
@@ -439,7 +439,10 @@ def run_optimization(
) )
) )
return ems_eos.genetic_solution().model_dump_json() solution = ems_eos.genetic_solution()
if solution is None:
return None
return solution.model_dump_json()
def main(): def main():