Improve caching. (#431)

* Move the caching module to core.

Add an in memory cache that for caching function and method results
during an energy management run (optimization run). Two decorators
are provided for methods and functions.

* Improve the file cache store by load and save functions.

Make EOS load the cache file store on startup and save it on shutdown.
Add a cyclic task that cleans the cache file store from outdated cache files.

* Improve startup of EOSdash by EOS

Make EOS starting EOSdash adhere to path configuration given in EOS.
The whole environment from EOS is now passed to EOSdash.
Should also prevent test errors due to unwanted/ wrong config file creation.

Both servers now provide a health endpoint that can be used to detect whether
the server is running. This is also used for testing now.

* Improve startup of EOS

EOS now has got an energy management task that runs shortly after startup.
It tries to execute energy management runs with predictions newly fetched
or initialized from cached data on first run.

* Improve shutdown of EOS

EOS has now a shutdown task that shuts EOS down gracefully with some
time delay to allow REST API requests for shutdwon or restart to be fully serviced.

* Improve EMS

Add energy management task for repeated energy management controlled by
startup delay and interval configuration parameters.
Translate EnergieManagementSystem to english EnergyManagement.

* Add administration endpoints

  - endpoints to control caching from REST API.
  - endpoints to control server restart (will not work on Windows) and shutdown from REST API

* Improve doc generation

Use "\n" linenend convention also on Windows when generating doc files.
Replace Windows specific 127.0.0.1 address by standard 0.0.0.0.

* Improve test support (to be able to test caching)

  - Add system test option to pytest for running tests with "real" resources
  - Add new test fixture to start server for test class and test function
  - Make kill signal adapt to Windows/ Linux
  - Use consistently "\n" for lineends when writing text files in  doc test
  - Fix test_logging under Windows
  - Fix conftest config_default_dirs test fixture under Windows

From @Lasall

* Improve Windows support

 - Use 127.0.0.1 as default config host (model defaults) and
   addionally redirect 0.0.0.0 to localhost on Windows (because default
   config file still has 0.0.0.0).
 - Update install/startup instructions as package installation is
   required atm.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>

src/akkudoktoreos/server/eosdash.py

@@ -1,11 +1,14 @@
 import argparse
 import os
 import sys
+import time
 from functools import reduce
 from typing import Any, Union
 
+import psutil
 import uvicorn
 from fasthtml.common import H1, Table, Td, Th, Thead, Titled, Tr, fast_app
+from fasthtml.starlette import JSONResponse
 from pydantic.fields import ComputedFieldInfo, FieldInfo
 from pydantic_core import PydanticUndefined
 
@@ -121,6 +124,17 @@ def get():  # type: ignore
     return Titled("EOS Dashboard", H1("Configuration"), config_table())
 
 
+@app.get("/eosdash/health")
+def get_eosdash_health():  # type: ignore
+    """Health check endpoint to verify that the EOSdash server is alive."""
+    return JSONResponse(
+        {
+            "status": "alive",
+            "pid": psutil.Process().pid,
+        }
+    )
+
+
 def run_eosdash(host: str, port: int, log_level: str, access_log: bool, reload: bool) -> None:
     """Run the EOSdash server with the specified configurations.
 
@@ -131,20 +145,54 @@ def run_eosdash(host: str, port: int, log_level: str, access_log: bool, reload:
     server to the specified host and port, an error message is logged and the
     application exits.
 
-    Parameters:
-    host (str): The hostname to bind the server to.
-    port (int): The port number to bind the server to.
-    log_level (str): The log level for the server. Options include "critical", "error",
-                     "warning", "info", "debug", and "trace".
-    access_log (bool): Whether to enable or disable the access log. Set to True to enable.
-    reload (bool): Whether to enable or disable auto-reload. Set to True for development.
+    Args:
+        host (str): The hostname to bind the server to.
+        port (int): The port number to bind the server to.
+        log_level (str): The log level for the server. Options include "critical", "error",
+                        "warning", "info", "debug", and "trace".
+        access_log (bool): Whether to enable or disable the access log. Set to True to enable.
+        reload (bool): Whether to enable or disable auto-reload. Set to True for development.
 
     Returns:
-    None
+        None
     """
     # Make hostname Windows friendly
     if host == "0.0.0.0" and os.name == "nt":
         host = "localhost"
+
+    # Wait for EOSdash port to be free - e.g. in case of restart
+    timeout = 120  # Maximum 120 seconds to wait
+    process_info: list[dict] = []
+    for retries in range(int(timeout / 3)):
+        process_info = []
+        pids: list[int] = []
+        for conn in psutil.net_connections(kind="inet"):
+            if conn.laddr.port == port:
+                if conn.pid not in pids:
+                    # Get fresh process info
+                    process = psutil.Process(conn.pid)
+                    pids.append(conn.pid)
+                    process_info.append(process.as_dict(attrs=["pid", "cmdline"]))
+        if len(process_info) == 0:
+            break
+        logger.info(f"EOSdash waiting for port `{port}` ...")
+        time.sleep(3)
+    if len(process_info) > 0:
+        logger.warning(f"EOSdash port `{port}` in use.")
+        for info in process_info:
+            logger.warning(f"PID: `{info["pid"]}`, CMD: `{info["cmdline"]}`")
+
+    # Setup config from args
+    if args:
+        if args.eos_host:
+            config_eos.server.host = args.eos_host
+        if args.eos_port:
+            config_eos.server.port = args.eos_port
+        if args.host:
+            config_eos.server.eosdash_host = args.host
+        if args.port:
+            config_eos.server.eosdash_port = args.port
+
     try:
         uvicorn.run(
             "akkudoktoreos.server.eosdash:app",
@@ -197,13 +245,13 @@ def main() -> None:
         "--eos-host",
         type=str,
         default=str(config_eos.server.host),
-        help="Host for the EOS server (default: value from config)",
+        help="Host of the EOS server (default: value from config)",
     )
     parser.add_argument(
         "--eos-port",
         type=int,
         default=config_eos.server.port,
-        help="Port for the EOS server (default: value from config)",
+        help="Port of the EOS server (default: value from config)",
     )
 
     # Optional arguments for log_level, access_log, and reload
@@ -230,7 +278,9 @@ def main() -> None:
 
     try:
         run_eosdash(args.host, args.port, args.log_level, args.access_log, args.reload)
-    except:
+    except Exception as ex:
+        error_msg = f"Failed to run EOSdash: {ex}"
+        logger.error(error_msg)
         sys.exit(1)