Wire-format parity fixes in reader.py for trailing fields the
companion-radio firmware emits but the SDK decoder was dropping, plus an
over-read guard on DEFAULT_FLOOD_SCOPE. Each fix uses the existing
BATT_AND_STORAGE defensive-read pattern (up-front minimum-length check
plus per-field cumulative-length gates), so older firmware that doesn't
emit the trailing fields keeps decoding without raising. Every fix ships
a legacy-frame and modern-frame unit test pair.
AUTOADD_CONFIG (PacketType 25): companion-v1.14.0 firmware emits a
trailing max_hops byte (firmware commit 00566741); the SDK dropped it.
Adds a defensive `if len(data) >= 3` read. Pre-v1.14.0 frames decode to
`{"config": ...}` with no max_hops key.
LOGIN_SUCCESS (PacketType 0x85): the new-style RESP_SERVER_LOGIN_OK path
emits three trailing fields the SDK was dropping — server_timestamp (4B,
firmware commit 0e90b731), acl_permissions (1B, 7947e8a2), and
fw_ver_level (1B, 418ae08b), all first shipped in companion-v1.10.0.
Adds per-field length gates (>=12, >=13, >=14). Legacy 8-byte "OK"-path
frames decode unchanged.
ACK (PacketType 0x82): firmware emits a trailing 4-byte trip_time
(round-trip latency in ms) since companion-v1.0.0a (firmware commit
d9dc76f1, Jan 2025) — on the wire ~16 months but never surfaced by the
SDK. Adds an `if len(data) >= 9` read. Legacy 5-byte ACK frames decode
unchanged.
DEFAULT_FLOOD_SCOPE (PacketType 28): firmware emits a 48-byte populated
frame or a 1-byte sentinel when no scope is set. The SDK unconditionally
read 31+16 bytes, over-reading 47 bytes past the end of the sentinel
frame and dispatching `{"scope_name": "", "scope_key": ""}`. Adds an
`if len(data) >= 48` guard so the sentinel dispatches `{}`. Consumers
detecting "no scope" via `payload["scope_name"] == ""` should switch to
a key-presence check.
RAW_DATA: the payload-framing fix this branch originally carried landed
upstream first in PR #86 (commit 44b21be), with identical decode logic
(discard the reserved 0xFF byte, then read the remaining variable-length
payload). That redundant change is dropped here; this commit retains only
its regression test (test_raw_data_realistic_frame), which exercises the
upstream fix end-to-end — the upstream change shipped without a test.
CHANGELOG:
- Decode max_hops trailing byte in AUTOADD_CONFIG (companion-v1.14.0+).
- Decode server_timestamp / acl_permissions / fw_ver_level trailing
fields in LOGIN_SUCCESS (companion-v1.10.0+).
- Decode trip_time trailing field in ACK (companion-v1.0.0a+).
- DEFAULT_FLOOD_SCOPE dispatches `{}` on the 1-byte sentinel frame
instead of `{scope_name: "", scope_key: ""}`. Detect "no scope" via
key presence.
Tests: 10 tests in tests/unit/test_protocol_surface_gaps.py (legacy +
modern frame pair per finding, including a RAW_DATA regression test for
the upstream fix); all 10 pass on the rebased upstream base.
Why: companion-radio firmware has been emitting these fields on the wire
for between 2 and 16 months; SDK consumers (integrations, bots,
dashboards) lose access to data that is on the wire today.
Why: companion-radio firmware emits RESP_CODE_CHANNEL_DATA_RECV (27) for
group-channel binary data (PAYLOAD_TYPE_GRP_DATA), shipped in
companion-v1.15.0. The SDK had no PacketType value 27, no EventType, and
no reader handler, so these frames hit the unknown-packet-type
fallthrough and the payload was silently dropped.
This adds PacketType.CHANNEL_DATA_RECV = 27 (the previously-skipped enum
slot), EventType.CHANNEL_DATA_RECV, and a reader handler. The fixed
9-byte header (snr, reserved, channel_idx, path_len with the
sentinel/hash-mode encoding) reuses CHANNEL_MSG_RECV_V3's framing; the
typed tail decodes data_type (uint16 little-endian, widened from uint8
in firmware), data_len, and payload (hex string). An up-front length
gate mirrors the defensive-read pattern used by the other handlers.
data_type is exposed as an int (matching the txt_type convention) and
payload as a hex string (matching RAW_DATA's convention for binary data
of unknown encoding); attributes surface channel_idx and data_type so
subscribers can filter without unpacking the payload.
Tests: tests/unit/test_protocol_surface_gaps.py — 5 new tests (enum
slot present, direct-path frame, route-flood path_len bit-split,
under-minimum frame dropped, widened data_type round-trip). Full unit
suite: 146 passed.
Rename test_g6_protocol_surface_gaps.py to test_protocol_surface_gaps.py.
Strip G6 from module docstring, and finding IDs (N01, N02, N03, N05,
N09, R04) from docstrings and section comments.
Rename test_g5_asyncio_lifecycle.py to test_asyncio_lifecycle.py.
Strip G5 from module docstring, finding IDs (F05, F07, F08, F19)
from class names and docstrings.
Rename test_g4_transport_symmetry.py to test_transport_symmetry.py.
Strip G4 from module docstring, _g4_ from function names, and finding
IDs (F04, NEW-A, F18, M06, N04) from docstrings and section comments.
Rename test_g2_error_handling.py to test_error_handling.py. Strip G2
prefix from module docstring, _g2_ from function names, and finding
IDs (F22, F21/M01, M02, M04, N06, F14) from docstrings and section
comments. Proposal cross-references removed.
Strip internal forensics finding references (F06, N07, NEW-C, R02)
from docstrings, comments, and assertion messages. Descriptive text
is preserved — only the ID prefixes are removed.
Strip internal forensics finding references (F01, F02, F03, N11)
from docstrings and section comments. The descriptive text is
preserved — only the ID prefixes are removed.
Strip G1/ prefix from docstrings, _g1_ from function names,
and G1 from section comments. Finding IDs (F06, N07, etc.)
are preserved as they are meaningful standalone references.
send_binary_req reads result.payload['expected_ack'] from MSG_SENT
events. The resolving subscribe must provide that key for MSG_SENT
event types to avoid KeyError.
The test_r03_placeholder_registered_before_send test used a bare
MagicMock dispatcher whose subscribe never resolved event futures,
causing send() to block for DEFAULT_TIMEOUT (15s). Add a resolving
subscribe mock matching the pattern from the fixture fix on
fix/test-timeout-waste.
The mock_dispatcher fixture's fake_subscribe recorded event handlers
but never called them, causing asyncio.wait() to block for the full
DEFAULT_TIMEOUT (15s) on every test that passes expected_events to
send(). With 28 affected tests, the suite wasted ~8 minutes on dead
waits and required an undocumented pytest-timeout plugin to complete.
Add call_soon to the default fake_subscribe so futures resolve on the
next event loop iteration, matching the pattern already used by
setup_event_response(). Override with a non-resolving mock in
test_send_timeout to preserve timeout path coverage.
Suite now completes in <1 second with no --timeout flag.
Why: MeshCore.subscribe typed callbacks as Callable[[Event], Coroutine[...]]
(async only), while EventDispatcher.subscribe typed them as
Callable[[Event], Union[None, asyncio.Future]] (sync or async). Type-checkers
flag any sync handler passed through MeshCore.subscribe. Fix: align the
annotation to match EventDispatcher's union type; remove unused Coroutine import.
Refs: Forensics report finding R05
Why: send_binary_req() registered the pending request with the reader AFTER
send() returned. If a BINARY_RESPONSE arrives between send() returning and
registration (reachable for TCP-companion proxies), the reader logs "No tracked
request found" and the caller's wait_for_event times out. Fix: pre-register a
placeholder keyed by object id before send(), then swap it for the real tag
from MSG_SENT. On send() failure, the placeholder is cleaned up.
Refs: Forensics report finding R03
Why: Three minor cleanup fixes. M03 adds an else branch in set_flood_scope so
unsupported scope types raise TypeError instead of UnboundLocalError. M05 removes
the dead `out_path_len >> 6` shift in update_contact (high bits always zero due
to reader masking) and initializes path_hash_mode=0 explicitly. M07 normalizes
three `return None` paths in get_contacts to return Event(EventType.ERROR, ...)
so callers can rely on the return type always being Event.
Refs: Forensics report findings M03, M05, M07
Why: 5 seconds is too short for slow-path mesh operations (path-resolving
messaging, long binary responses, remote auth). Also the root cause of tests
that appeared to "hang" — they were falling through to the 5s timeout because
their mock dispatchers don't wire matching responses. Landed as a separate
commit so reviewers can drop it independently if they push back.
Refs: Forensics report finding F09
Why: req_mma() references undefined variables `start` and `end`, causing a
NameError on every call. The logger.error migration warning confirms the method
is intentionally deprecated in favor of req_mma_sync. Since it is broken as
shipped, removing it cannot break any working caller.
Refs: Forensics report finding F13
Why: Five CommandType enum entries had no user-facing SDK method:
SEND_RAW_DATA (25), HAS_CONNECTION (28), GET_CONTACT_BY_KEY (30),
GET_TUNING_PARAMS (43), FACTORY_RESET (51). Added send_raw_data() in
MessagingCommands, has_connection()/get_tuning()/request_factory_reset()/
confirm_factory_reset() in DeviceCommands, get_contact_by_key() in
ContactCommands. FACTORY_RESET uses a two-step token pattern as a
Python-side safety measure against accidental invocation.
Refs: Forensics report finding N09 (also N03 for get_tuning)
Why: Firmware requires strict len > 10 (MyMesh.cpp:1620). When path is
empty, send_trace() builds exactly 10 bytes (cmd+tag+auth+flags), which
is silently rejected. Appending one zero byte when the packet is <= 10
bytes satisfies the firmware gate without changing the semantic content.
Refs: Forensics report finding N05
Why: Three firmware push/response codes had no SDK handler — frames fell
through to the "Unhandled" debug log. CONTACT_DELETED (0x8F) carries a
32-byte pubkey from onContactOverwrite(); CONTACTS_FULL (0x90) is a
1-byte push from onContactsFull(); TUNING_PARAMS (23) is the 9-byte
response to CMD_GET_TUNING_PARAMS carrying rx_delay and airtime_factor.
All three now dispatch typed events. Short frames are guarded.
Refs: Forensics report findings N01, N02, N03
Why: PacketType was missing CONTACTS_FULL (0x90), emitted by
MyMesh::onContactsFull(). CommandType was missing GET_STATS (56),
used by get_stats_core/radio/packets but referenced only as literal
b"\\x38". Adding both enum entries prepares for handler and wrapper
implementations in subsequent commits.
Refs: Forensics report findings N02, R04
10 new tests in tests/unit/test_g5_asyncio_lifecycle.py:
- TestF05: _spawn_background retains tasks in TCP, Serial, and
EventDispatcher; tracked tasks survive gc.collect(); TCP handle_rx
and connection_lost use tracked dispatch.
- TestF07: stop() waits for in-flight async callbacks to complete.
- TestF08: EventDispatcher.queue is None before start(), created on
start(), dispatch() before start() raises RuntimeError;
CommandHandlerBase lock is None before access, created lazily.
- TestF19: send() calls get_running_loop (not get_event_loop).
Refs: Forensics report findings F05, F07, F08, F19
Why: asyncio.get_event_loop() inside an async function emits
DeprecationWarning since Python 3.10 and raises in some contexts on
Python 3.12+. The call in CommandHandlerBase.send() is always inside
a running async context where get_running_loop() is the correct API.
Refs: Forensics report finding F19
Why: On Python 3.9/3.10, asyncio.Queue() and asyncio.Lock() bind to
the running event loop at construction time. If the SDK is instantiated
from a synchronous factory before an event loop exists, both primitives
raise "RuntimeError: ... is bound to a different event loop" on first
use. Fix: EventDispatcher defers Queue creation to start(), with a
guard in dispatch() that raises RuntimeError if called before start().
CommandHandlerBase defers Lock creation via a lazy @property accessor.
Both document the contract change in class docstrings.
Refs: Forensics report finding F08
Why: EventDispatcher._process_events() calls task_done() on the queue
immediately after spawning async callback tasks. await queue.join() in
stop() therefore returns as soon as all items are marked done, even if
their async callbacks are still executing. Any caller that does
"await dispatcher.stop(); cleanup()" could race with still-running
callbacks. Fix: after queue.join(), gather all tracked background tasks
before cancelling the dispatch loop.
Refs: Forensics report finding F07
Why: Python's asyncio holds only weak references to tasks created via
create_task(). Under GC pressure (especially Python < 3.11), unretained
tasks can be silently cancelled mid-execution, and any exceptions are
swallowed as "Task exception was never retrieved." Seven call sites
across TCPConnection, BLEConnection, SerialConnection, and
EventDispatcher used fire-and-forget create_task with no stored
reference. Fix: introduce _background_tasks set and _spawn_background()
helper on each class, following the standard pattern from the asyncio
docs (task.add_done_callback(set.discard)).
Refs: Forensics report finding F05
