fix(genetic): keep fitness-cache memory within pymalloc and release arena after each run (#1353)

* fix(genetic): keep fitness-cache memory in pymalloc and release arena after each run

At fine time resolution (interval_sec=900, ~192 control slots over a multi-day
horizon) the fitness-cache keys are ~1.6 KB int tuples, above CPython's 512-byte
pymalloc threshold, so they are served by glibc malloc in the optimization worker
thread's arena and are not returned to the OS on `self._fitness_cache.clear()`.
With re-optimization every 15 min, RSS stair-steps up to the memory limit within
about a day (OOM / forced restart). At hourly resolution the tuples stay < 512 B,
so pymalloc reclaims them and the effect is negligible. See #1352.

- Store the cache key/genome compactly as bytes (1 byte per gene, 8-byte fallback
  for larger state spaces) so entries stay within pymalloc regardless of resolution.
- After each optimization run, gc.collect() + malloc_trim(0) (guarded, glibc-only)
  to return freed arena pages to the OS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(genetic): pack fitness-cache genome as bounded chunks

Storing the genome as a single bytes object still exceeds pymalloc's
512-byte threshold once the genome grows: at 15-min resolution over a
60 h horizon with EV genes the key is ~480 genes, so even the one-byte
encoding is 481 bytes (514 with the object header) and any value >255
switches the whole genome to 8 bytes per gene. Such keys land in glibc
malloc, which does not reliably return the pages (malloc_trim is
glibc-only, absent on musl) — the platform-independent guarantee did
not actually hold for supported settings.

Encode the genome (key and FitnessCacheEntry.genome) as a tuple of
bounded byte chunks instead — 256 one-byte genes or 32 signed-64-bit
genes per chunk, 256 bytes each — built per chunk so the encoder never
materialises an oversized temporary. Every object then stays inside
pymalloc regardless of horizon, on every platform. malloc_trim after a
run is kept as a secondary release for the rest of the run's heap.

Tests: round-trips incl. 480-gene narrow/wide and negative genes, and
sys.getsizeof for the key AND every chunk at 480 genes in both
encodings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

tests/test_genetic_seeding.py

@@ -1,3 +1,4 @@
+import sys
 from types import SimpleNamespace
 from unittest.mock import patch
 
@@ -7,7 +8,12 @@ from deap import creator, tools
 
 from akkudoktoreos.config.config import ConfigEOS
 from akkudoktoreos.core.coreabc import get_ems
-from akkudoktoreos.optimization.genetic.genetic import GeneticOptimization
+from akkudoktoreos.optimization.genetic.genetic import (
+    GeneticOptimization,
+    _pack_genes,
+    _release_freed_memory,
+    _unpack_genes,
+)
 from akkudoktoreos.utils.datetimeutil import to_datetime
 
 
@@ -56,6 +62,53 @@ def test_ev_repair_is_resimulated_before_fitness_assignment(config_eos: ConfigEO
     assert individual[opt.control_slots :] == [0] * opt.control_slots
 
 
+@pytest.mark.parametrize(
+    "genes",
+    [
+        [],
+        [0, 1, 255],
+        [3] * 192,
+        [0, 256, 1],
+        [70000, 2, 0],
+        [7] * 480,  # 60 h at 15 min with EV genes, all one-byte
+        [7] * 479 + [256],  # same length, one value forces the 8-byte encoding
+        [-1, 0, 300],  # negatives also take the signed 8-byte path
+    ],
+)
+def test_pack_genes_roundtrip(genes: list[int]):
+    assert _unpack_genes(_pack_genes(genes)) == genes
+
+
+@pytest.mark.parametrize(
+    "genes",
+    [
+        [7] * 480,  # narrow: ~60 h at 15 min with EV genes, all one-byte
+        [7] * 479 + [256],  # wide: one value forces 8 bytes for the whole genome
+    ],
+)
+def test_pack_genes_key_and_chunks_stay_below_pymalloc_limit(genes: list[int]):
+    # A 60 h/15 min horizon with EV genes reaches ~480 genes. The packed key and
+    # every chunk must stay small objects (<= 512 B), otherwise ~100k cache
+    # entries per run go to glibc malloc and are never given back to the OS.
+    packed = _pack_genes(genes)
+    assert sys.getsizeof(packed) <= 512
+    for chunk in packed:
+        assert sys.getsizeof(chunk) <= 512
+    assert _unpack_genes(packed) == genes
+
+
+def test_pack_genes_encodings_do_not_collide():
+    small = _pack_genes([1, 0, 0, 0, 0, 0, 0, 0])
+    wide = _pack_genes([1, 256])
+    assert _pack_genes([1]) != _pack_genes([1, 0])
+    assert small != wide
+    assert _unpack_genes(wide) == [1, 256]
+
+
+def test_release_freed_memory_is_safe_to_call():
+    _release_freed_memory()
+
+
 def test_fitness_cache_restores_canonical_ev_genome(config_eos: ConfigEOS):
     _configure_hourly_grid(config_eos)
     opt = GeneticOptimization(fixed_seed=42)