mirror of
https://github.com/Akkudoktor-EOS/EOS.git
synced 2026-07-24 18:58:12 +00:00
fix: db compaction run (#1156)
These fixes were taken from https://github.com/arneman/EOS/tree/refactor/economic-objective. 1. Fix compaction job registration bug in eos.py: - compact_eos_database was registered with save_eos_database function - Now correctly calls compact_eos_database() - Root cause: compaction/vacuum never ran, allowing records to grow unbounded 2. Optimize db_iterate_records() from O(n) to O(log n): - Previous: linear scan from index 0 for every key_to_array() call - Now: uses bisect_left on _db_sorted_timestamps to skip to start position - Critical for large datasets: 9400 measurement records → 94s/call overhead → 5 calls/optimization 3. Reduce compaction_interval_sec default from 604800s (7 days) to 3600s (1 hour): - With 7-day interval: 65000 records accumulate before first cleanup - Bridge pushes ~1 record/9s (grid_export_emr) - Hourly compaction + 2h data window → stable ~950 records Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
This commit is contained in:
10
openapi.json
10
openapi.json
@@ -8,7 +8,7 @@
|
||||
"name": "Apache 2.0",
|
||||
"url": "https://www.apache.org/licenses/LICENSE-2.0.html"
|
||||
},
|
||||
"version": "v0.3.0.dev2607181096091018"
|
||||
"version": "v0.3.0.dev2607200446185246"
|
||||
},
|
||||
"paths": {
|
||||
"/v1/admin/cache/clear": {
|
||||
@@ -2959,9 +2959,9 @@
|
||||
],
|
||||
"title": "Compaction Interval Sec",
|
||||
"description": "Interval in between automatic tiered compaction runs [seconds].\nCompaction downsamples old records to reduce storage while retaining coverage. Set to None to disable automatic compaction.",
|
||||
"default": 604800,
|
||||
"default": 3600,
|
||||
"examples": [
|
||||
604800
|
||||
3600
|
||||
]
|
||||
},
|
||||
"batch_size": {
|
||||
@@ -3070,9 +3070,9 @@
|
||||
],
|
||||
"title": "Compaction Interval Sec",
|
||||
"description": "Interval in between automatic tiered compaction runs [seconds].\nCompaction downsamples old records to reduce storage while retaining coverage. Set to None to disable automatic compaction.",
|
||||
"default": 604800,
|
||||
"default": 3600,
|
||||
"examples": [
|
||||
604800
|
||||
3600
|
||||
]
|
||||
},
|
||||
"batch_size": {
|
||||
|
||||
Reference in New Issue
Block a user