External Reticulum execution¶
Use Reticulum → Radio → Board to choose a USB serial port, an encrypted
Classic RFCOMM endpoint (rfcomm://ADDRESS/CHANNEL), or a private local owner
socket (unix:///absolute/path). Disconnect before changing the board or mode.
The socket can be an existing encrypted SSH forward. Raw TCP is not accepted.
Choose Auto to prefer a ready external executor and use the local backend otherwise. Disabled uses the local backend without polling the board. External only prevents local fallback. Connect once; Auto subsequently switches when the node becomes ready, disappears, or returns. Detection polls once every 20 seconds; executor event/lease failures trigger recovery sooner. Refresh reads status immediately. No configured endpoint means no node scanning.
A reachable modem is not necessarily an executor. Legacy schema-2 provisioning reports identity and radio settings but cannot accept Pocket links, resources or LXMF endpoints. Auto keeps those applications local until the complete API below is advertised. A former USB RNode port can be selected as the external endpoint. That port is then reserved for node control: local fallback needs a separate radio. Pocket will report it unavailable rather than send RNode setup commands to the executor. The first successfully observed board identity is pinned. A replacement identity requires explicit Board reselection.
The host keeps the application identity across handoffs. It drains outstanding service operations, stops the old runtime, and starts the selected backend through the existing NativeBackend and service bridge. New commands during a handoff are rejected. Interrupted commands are never replayed: their result can be unknown. External-only loss reports an error instead of silently starting a local radio. Explicit sharing grants and discovery settings follow automatic backend changes within the same controller session. The new runtime validates those grants again before reporting ready. Open apps resubscribe through the existing provider when a backend replacement ends their subscription. Revocation removes grants, and an explicit disconnect clears them. Messages and service commands are never replayed. A ready node returning to either Auto or External only wakes the existing connection-recovery loop. Missing local fallback configuration leaves management available and does not prevent recovery when the external node returns. Disposable session sockets use the same ownership receipts and lifetime locks as other local services. A dead socket is removed only when its receipt still matches, no owner holds its lifetime lock, and no listener accepts a connection. Unknown, replaced, live or locked sockets remain untouched.
Executor contract, version 1¶
The host, integration-test executor and merged Heltec firmware implement this contract. Board firmware must pass the physical Auto, service and messaging acceptance tests before deployment. The software fixture is for host tests only and must not be installed on a board.
USB/Classic provisioning retains KISS command 0x86 and reply 0x87, carrying
MessagePack [operation, sequence, data]. GET_INFO operation 2 may include key
128 with this metadata (string keys):
protocol: 1
abi: 2
identity: <32 lowercase hexadecimal characters identifying the board>
features: [links, endpoints, resources, lxmf]
lease_seconds: <number from 1 through 5>
online: true
available: true # optional: free or owned by this physical connection
framing: kiss-fragment-v1 # USB/Classic; raw PS02 remains valid for local IPC
fragment_size: 1024
clock: unix-ms-v1 # optional: accepts host wall clock at native_open
All four features and a ready radio interface are required. The metadata identity
must equal the existing provisioning transport identity. A private Unix executor
returns the same metadata from the node_info owner action.
When clock: unix-ms-v1 is advertised, native_open includes unix_ms, the
host's Unix time in milliseconds. It must be a positive signed 64-bit integer;
booleans and floating-point values are rejected. This field is optional and
valid only on native_open. Hosts omit it for older executors. The executor
uses it for LXMF timestamps and announce emission times, while lease, routing
and request deadlines retain their monotonic clock.
KISS command 0x88 carries the existing PS02 owner protocol, including its bounded
binary framing. Each envelope adds an integer channel; replies and pushed
events echo that channel. Provisioning and owner traffic share one reader and
one serialized writer. Hosts allow at most eight channels, with bounded queues.
There is no JSON or native address on the wire.
Large PS02 frames use KISS command 0x89 when kiss-fragment-v1 is advertised.
Each fragment has an eight-byte network-order header: message ID (uint32),
offset (uint16), total PS02 frame length (uint16), followed by 1–1024 payload
bytes (or the negotiated size, bounded to 256–2048). Total length is at most
32768. Offsets must be contiguous within each message. Receivers retain at most
eight assemblies and discard partial messages after ten seconds without a chunk.
No operation executes before reassembly and normal PS02 validation complete.
Renewals, cancellations, callback replies and provisioning can preempt bulk transfers between complete fragments. Bytes from different frames never interleave. A transport close aborts remaining fragments without replay. This keeps control requests responsive even with worst-case escaped binary payloads; it does not increase lease durations or write timeouts.
available: false means another physical client holds the context. The current
lease owner's probes must report true. Auto keeps execution local while another
client owns the board; native_open still enforces exclusivity against races.
Status and radio metrics remain readable when capabilities or interfaces are
not ready, independently of application-offload readiness.
Owner requests contain version: 1, action and, after opening, session (a
random 32-character host session identifier). Implement these actions:
| Action | Request fields | Response |
|---|---|---|
native_open |
identity, abi: 2, session |
Metadata above; claim an exclusive endpoint lease |
native_call |
operation, arguments |
result integer, outputs byte arrays; negative result includes native_error |
native_cancel |
session |
Cancel stamp computation without waiting on the native worker lock; integer result |
native_renew |
session |
Acknowledge lease extension |
subscribe |
session |
Stream session, ready: true, then event envelopes |
native_callback |
token, data bytes |
Acknowledge synchronous callback result |
native_call uses the pinned Pocket native ABI 2. The authoritative signatures
are signature_types in native_stack.py; the shared wire operation allowlist,
input/output buffer positions, capacities and strings are in native_rpc.py.
An input buffer becomes bytes; an output buffer becomes a null argument and its
bytes return in outputs, in declaration order. Fixed outputs must have exactly
the declared size. Variable outputs must match result and stay within capacity.
Buffers are limited to 32768 bytes. Handler allowlists contain 16 bytes per peer,
so their count argument is a peer count, not a byte count.
init carries [null, session_bytes]. The executor supplies its own callback and
storage location; a host filesystem path is never used on the board. All other
operations preserve ABI semantics, including identity import/export, destination
registration, links, request/reply, resources and native microLXMF. Existing
Pocket permissions, contact policy and service grants remain enforced by the
host. The executor owns actual radio transmission, reception and core polling;
the host consumes pushed events rather than polling the board at display speed.
Push events contain session, native_event (the existing ABI MessagePack event
bytes), and an optional callback token. Resource-advertisement callbacks await
native_callback; that action must remain responsive while a native worker is
waiting. The executor handles outgoing-radio callbacks locally. Cancellation,
lease renewal and callback replies must not queue behind expensive stamp work.
A lost/reset context must stop transmitting before its lease expires. Reject
requests from expired or other sessions. Successful ABI reset acknowledges
release. Without that acknowledgement the host closes all pending channels,
stops renewing, waits a full negotiated lease interval after closing, then permits local
replacement. Scope release to the host's context: do not reset an independently
running board router, IBSS service, or another owner's endpoints. Queued work
must not extend the lease or execute after expiry.
The configured executor receives application private identity material to act as those endpoints. Use only a trusted board. Do not expose this API to anonymous network peers, log key buffers, or persist imported keys beyond the selected storage policy. Classic RFCOMM requests Bluetooth medium security (encrypted link); failure to establish it fails the connection. Local sockets must be private and user-owned; remote forwarding must remain encrypted. USB relies on physical attachment trust.
Acceptance¶
Run from the repository with its native library and RNS/LXMF test environment:
python tests/integration/auto_runtime.py
POCKET_TEST_NATIVE_ROLE=a POCKET_TEST_EXTERNAL_ROLE=a python tests/integration/service_mesh.py
POCKET_TEST_NATIVE_ROLE=b POCKET_TEST_EXTERNAL_ROLE=b python tests/integration/service_mesh.py
tests/fixtures/native_executor.py executes the real native library in a
separate process. Auto acceptance covers local → external → local on loss →
external on return, identity continuity, Disabled, and External-only refusal.
The mesh test exercises the existing service bridge against stock RNS/LXMF,
including resources, subscriptions, grants/revocation and signed messaging.
tests/test_native_rpc.py checks binary framing, interleaved provisioning/RPC,
buffer bounds and interrupted requests. These are host acceptance tests; radio,
board memory, encrypted RFCOMM and hardware lease-expiry acceptance remain
required before claiming physical offload is ready.
Physical acceptance (exclusive board access)¶
tests/integration/hardware_auto.py takes --endpoint, --fallback-config and
--evidence. Add --peer-socket for end-to-end acceptance: it requires a
real service call and a signed LXMF message/receipt in both directions in every
phase, with the offloaded message timestamp matching the host wall clock. Without a
peer it checks connectivity only. Run it with the native library and real executor
firmware available. The fallback configuration selects one native TCP or RNode
interface; an RNode configuration must preserve the existing radio settings.
Stop any other owner of that radio first. This runner uses isolated application
identities and temporary state. It deliberately closes the external connection,
checks local takeover, permits reconnection, and requires the same application
identity after return. It does not reboot or flash the board. It restores only
its own test connection; it does not manage production owners.
Unexpected executor failures fail the run even if automatic recovery later
succeeds. The report separates backend readiness from service and signed-message
completion times; the deliberate connection loss and final shutdown are excluded
from unexpected failures.
For the service suite, POCKET_TEST_EXTERNAL_ENDPOINT selects the physical
executor instead of the fixture. POCKET_TEST_HARDWARE_INTERFACE names an
additional stock-RNS interface configuration section for the test hub, providing
the actual packet path to that board. Use it with the existing native/external
role variables. Hold exclusive USB/RFCOMM access for the entire test. Success
requires the same service and LXMF assertions as the simulated-executor test;
status queries alone do not pass acceptance.
Portable native-call dispatcher¶
Firmware can include shell/native/reticulum/rpc_dispatch.h and call:
auto result = pocket_rpc::dispatch(operation, encoded_arguments, length,
pocket_rpc::Context{event_callback, executor_storage});
operation is a std::string and encoded_arguments is the standalone binary
MessagePack argument array, without PS02 headers. Result owns result,
outputs (byte arrays), and native_error. pack_result(packer, result) writes
those fields plus version: 1; the transport adds its channel/session envelope.
Argument spans returned by Reader(raw, length).read() borrow the input frame,
which must remain alive until the operation completes. No pointer is decoded
from the wire. init requires a null callback placeholder and uses the injected
callback/storage, discarding the second wire argument as a filesystem location.
The dispatcher owns validation and ABI calls only. Firmware still owns session
checks, leases, scheduling, event callbacks and asynchronous stamp work. It may
intercept lxmf_send before invoking the synchronous native operation. No OS
threads or filesystem operations are introduced by the header.
Build the boundary tests and an optional bridge to the real native library with:
python tests/integration/native_rpc_dispatch.py --native-build /path/to/pinned/native/build --bridge-output /tmp/pocket-rpc-dispatch.so
POCKET_TEST_CPP_DISPATCH=/tmp/pocket-rpc-dispatch.so POCKET_TEST_NATIVE_ROLE=a POCKET_TEST_EXTERNAL_ROLE=a python tests/integration/service_mesh.py
Use .dylib for the bridge on macOS. The first command checks every operation,
invalid/truncated arguments, buffer/result bounds and ignored host storage paths,
then runs 100,000 decoder mutations with address/undefined-behavior sanitizers.
The second uses that compiled dispatcher with the real native core and stock
RNS/LXMF peers. These checks require a C++17 compiler and an already prepared
native CMake build; they access no physical device.
The companion request_decode.h validates the outer owner map after transport
framing has been checked:
auto request = pocket_rpc::decode_request(body, length);
auto result = pocket_rpc::dispatch(request.operation,
request.arguments.data, request.arguments.size, context);
Request owns the action/session/identity/operation/token strings and exposes
integer channel/abi. Its arguments and data spans borrow the immutable
request body until execution completes. Invalid input throws
std::invalid_argument; decoding never executes an ABI operation. Firmware must
check its lease/session before dispatching the decoded operation.
Only the six native actions and their exact field sets are accepted. The map is limited to 16 string-key fields, with no duplicates or trailing data. Version is 1, channel is a positive int32, and session/identity are 32 lowercase hexadecimal characters. Arguments remain a validated flat MessagePack array; callback data is binary. Unknown fields, nested objects, wrong types and oversized strings are rejected. Native boundary tests also cover every truncation of valid maps and 100,000 mutations of valid outer requests, in addition to argument mutations.
An isolated stock reference can be started on the laptop with
hardware_auto.py --stock-peer SOCKET STATE CONFIG_DIRECTORY. Its native/UDP
interface configuration must be explicitly enabled. The handset client connects
to its private owner socket through existing SSH forwarding when needed; it
imports no Python RNS or LXMF. Use temporary state for both test identities.