Skip to content

fix: emit mq payloads as standalone arrays, not views into the packet buffer - #402

Merged
gdoumen merged 1 commit into
mainfrom
fix/mq-payload-copy
Aug 19, 2026
Merged

fix: emit mq payloads as standalone arrays, not views into the packet buffer#402
gdoumen merged 1 commit into
mainfrom
fix/mq-payload-copy

Conversation

@gdoumen

@gdoumen gdoumen commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

A feature toggle published to a device was rejected as a malformed payload — even though the bytes were exactly {"value":true}:

message:feature-toggle sync: ignoring malformed payload,
topic:incyclist/features/IOS-DEV/TOGGLE_TEST,
payload:{0:123,1:34,2:118,3:97,4:108,5:117,6:101,7:34,8:58,9:116,10:114,11:117,12:101,13:125}

Those 14 bytes decode to {"value":true}. The data arrived intact; reading it was the problem.

Cause

mqtt-packet builds a payload by slicing the stream buffer (parser.js:323), so what MQTT.js delivers is a view: byteOffset is non-zero and message.buffer is the whole packet, not the message.

Both services-side handlers — appstate/toggle-sync.ts:159 and activities/active-rides/mq.ts:149 — read the payload as:

str = Buffer.from(message.buffer).toString()

Buffer.from(arrayBuffer) wraps the entire buffer, ignoring byteOffset/byteLength. Reproduced locally: for a 14-byte payload at offset 14, that call returned the whole 8KB backing store including unrelated memory. JSON.parse then failed and the toggle was discarded.

Why it is fixed in the binding, not in services

That services code is unchanged and worked before, because every other binding hands over a standalone array: the native module this replaced did, and desktop's does because Electron IPC copies on the way through. Mobile was the only one exposing a view into a larger buffer.

Normalising here also fixes both consumers at once and needs no services release.

The fix

this.emit('mq-message', topic, new Uint8Array(message));

new Uint8Array(view) copies element-wise into an array that owns its buffer at offset 0. Payloads are small JSON documents, so the copy is cheap.

Buffer.from(message) would not work: Node pools small Buffer allocations, so the copy would itself sit at a non-zero offset in a shared pool — correct on device, where the buffer polyfill does not pool, but wrong under test. new Uint8Array is deterministic on both.

Scope

Active rides would have hit this tooactive-rides/mq.ts has the identical line. It was listed as unverified in TESTING_BACKLOG #58 precisely because no multi-user ride had been run over the new transport; this is what that would have found.

Test plan

  • npm test794 tests / 111 suites pass, including a new regression guard that feeds the binding a payload at a non-zero offset inside a larger packet and asserts the emitted array owns its buffer, then asserts the consumers' exact Buffer.from(payload.buffer).toString() pattern yields the message.
  • One existing assertion was relaxed: it checked payload.toString() returned 'hello', which pinned Buffer semantics the Uint8Array interface never promised. It now decodes the way consumers actually do.
  • npx tsc --noEmit — clean. npm run lint — 0 errors.

Not verified: not yet on a device. The check is to publish a feature toggle and confirm feature-toggle sync: applied toggle instead of ignoring malformed payload.

… buffer

A feature toggle published to a device was rejected as a malformed payload even
though its bytes were exactly `{"value":true}`. The bytes were fine; reading them
was not.

mqtt-packet builds a message payload by slicing the stream buffer
(`parser.js:323`), so what MQTT.js delivers is a *view*: `byteOffset` is non-zero
and `message.buffer` is the whole packet rather than the message. Both services-
side handlers - `appstate/toggle-sync.ts` and `activities/active-rides/mq.ts` -
read the payload as `Buffer.from(message.buffer).toString()`, which therefore
returned the entire backing store instead of the 14 bytes that were sent, and
JSON.parse failed.

That code is unchanged and worked before, because every other binding hands over
a standalone array: the native module this replaced did, and desktop's does
because Electron IPC copies on the way through. Mobile was the only one exposing
a view, so it is normalised here rather than making every consumer defend against
it - which also fixes both handlers at once and needs no services release.

`new Uint8Array(message)` copies element-wise into an array that owns its buffer
at offset 0. Note `Buffer.from(message)` would not do: Node's Buffer pools small
allocations, so the copy would itself be a view at a non-zero offset - correct on
device, where the polyfill does not pool, but wrong under test.

Active rides would have hit this too. It had been listed as unverified for
exactly this reason.
@sonarqubecloud

Copy link
Copy Markdown

@gdoumen
gdoumen marked this pull request as ready for review August 19, 2026 10:01
@gdoumen
gdoumen merged commit eeb597f into main Aug 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant