Skip to content

DEV-958: use the ST datasheet accel sensitivity for the gen-2 LSM6DSV - #288

Open
marknolan wants to merge 3 commits into
masterfrom
DEV-958_lsm6dsv_accel_sensitivity
Open

DEV-958: use the ST datasheet accel sensitivity for the gen-2 LSM6DSV#288
marknolan wants to merge 3 commits into
masterfrom
DEV-958_lsm6dsv_accel_sensitivity

Conversation

@marknolan

@marknolan marknolan commented Aug 20, 2026

Copy link
Copy Markdown
Member

SensorLSM6DSV derived accel sensitivity as 32768/(FS_g*9.80665), while the firmware seed, the web SDK catalog and the driver's own gen-1 classes all use the ST datasheet mg/LSB figure — a 0.0576% split, and the last such divergence in the gen-2 chain.

Range Before After
±2 g 1670.703 1671.665922915
±4 g 835.3517 835.832961457
±8 g 417.6759 417.916480729
±16 g 208.8379 208.958240364

This is a correctness fix — the old values were wrong

Note: an earlier version of this description argued the opposite — that ST's figures were merely 32768/FS rounded to three significant figures, making the old derived values the more precise ones, and this therefore a consistency trade-off. That was wrong. Corrected below and in the code (aa61da17).

AN5922 Table 27 ("Content of output data registers vs. acceleration", FS_XL = ±2 g) gives worked register contents, and they settle it:

Acceleration Registers LSB 0.061 mg/LSB predicts 32768/FS predicts
1 g 40h 09h 16393 16393.4 16384.0 ✗
350 mg 16h 69h 5737 5737.7 5734.4 ✗
−1 g BFh F7h −16393 −16393.4 −16384.0 ✗

The accelerometer does not map its nominal full scale onto exactly 2¹⁵ counts — it has a small margin, the same phenomenon as the gyro, just far smaller (0.058% against ~14.7%). So 0.061 mg/LSB is the genuine sensitivity and the old derivation was wrong by 0.0576%.

Corroborated four ways:

  1. The datasheet worked example above.
  2. ST's upstream driver STMicroelectronics/lsm6dsv-pid (master) — 0.061f / 0.122f / 0.244f / 0.488f, byte-identical to the copy the firmware already vendors.
  3. Every example in STMems_Standard_C_drivers/lsm6dsv_STdC/examples converts through the lsm6dsv_from_fsN_to_mg helpers; none derives 32768/FS.
  4. lsm6dsv_self_test.c uses from_fs4_to_mg, and self-test pass/fail limits are specified in mg — ST's own self-test only comes out right if 0.122 mg/LSB is the true sensitivity.

The gyro is deliberately left alone. Same phenomenon, much larger margin: 4.375 mdps/LSB implies a real full scale of 143.4 dps on the ±125 dps setting, exactly where the part is observed to saturate. That was the DEV-793 round-2 bug. The comments spell out the distinction so the two are not "reconciled" to each other later.

These values also match the firmware calibration seed, the web SDK catalog and the gen-1 SensorLSM6DS3/SensorLIS2DW12 classes, so the whole gen-2 chain now agrees.

Verification

  • 51 driver tests pass, including API_00008, whose CAL locks are updated (2 of its 3 accel assertions exceeded the 0.0001 tolerance, exactly as predicted).
  • ASM_PC_00005 against the regenerated references: 142 tests, 0 failures.
  • Scope check before regenerating: of 121 compared CSVs, 41 differ and all 41 are LSM6DSV accel. Accel2 (LIS2DW12), gyro, mag, GSR, light, skin-temp, non-wear and payload metadata are byte-identical.
  • Across 3,414,843 accel values, every changed value matches the predicted ratio — 78.11% exactly, 21.89% within one least-significant digit (expected, since the stored references are themselves rounded to 3 dp). Zero unexplained.

⚠️ Shared references regenerated

The 41 gen-2 accel reference CSVs in the SharePoint Data Repository have been regenerated. Branches built against a driver without this change will fail those ASM_PC_00005 gen-2 accel comparisons until the ASM_PC submodule pointer is bumped past this merge. Same transition as the DEV-922 regeneration. Old references backed up before overwrite.

Note on running the suite locally

ASM_PC_00005 currently needs JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF-8", otherwise a default Windows JVM writes the ± in the Sensor config header as cp1252 and all 26 gen-2 tests fail on that one header line. Already fixed and awaiting merge in ShimmerResearch/ASM_PC#365, which sets the charset explicitly at all five PrintWriter sites.

🤖 Generated with Claude Code

SensorLSM6DSV derived accel sensitivity as 32768/(FS_g*9.80665), while the
firmware seed, the web SDK catalog and the driver's own gen-1 classes all use
the ST datasheet mg/LSB figure - a 0.0576% split, and the only such divergence
left in the gen-2 chain.

  +-2 g   1670.703  -> 1671.665922915
  +-4 g   835.3517  -> 835.832961457
  +-8 g   417.6759  -> 417.916480729
  +-16 g  208.8379  -> 208.958240364

Worth being explicit that the old values were not a mistake, and this is not a
correctness fix. Checked against ST's own vendored driver (lsm6dsv_reg.c):
every printed accel figure - 0.061 / 0.122 / 0.244 / 0.488 mg/LSB - is exactly
32768/FS rounded to three significant figures, because the accel really does
span the full 16-bit range at nominal full scale. So the derived form was the
more precise of the two, and moving to the datasheet figure trades a little
accuracy for agreement.

Agreement is worth more here. The firmware seeds these values into the on-device
calibration blob, so they are what the device reports and what becomes the
source of truth once per-unit calibration is honoured; keeping the exact form
would leave a 0.0576% step change in calibrated output waiting to surface on
that day. 0.0576% is far inside per-unit sensitivity variation.

The gyro is a genuinely different case and is deliberately left alone: 4.375
mdps/LSB implies a real full scale of 143.4 dps on the +-125 dps setting, which
is where the part is observed to saturate, so there a 32768/FS derivation is
wrong by ~14.7% rather than by a rounding. The comment now spells out the
distinction so the two are not "fixed" to match each other later.

API_00008 CAL locks updated (accel +-4 g literal); 51 driver tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 20, 2026 14:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Aligns the gen-2 SensorLSM6DSV accelerometer sensitivity constants with the ST datasheet nominal mg/LSB figures to keep the Java driver consistent with firmware-seeded calibration defaults and other SDKs/components.

Changes:

  • Updated LSM6DSV accel sensitivity matrices (LSB per m/s²) to match datasheet-nominal mg/LSB-derived values, with expanded in-code rationale distinguishing accel vs gyro behavior.
  • Updated API_00008 regression-lock assertions and comments to reflect the new default accel sensitivity for ±4 g.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
ShimmerDriver/src/main/java/com/shimmerresearch/verisense/sensors/SensorLSM6DSV.java Switch accel sensitivity constants to datasheet-nominal-derived values and document the accel/gyro distinction.
ShimmerDriver/src/test/java/com/shimmerresearch/verisense/API_00008_VerisenseLsm6dsvTaggedFifoParsing.java Update regression-locked CAL assertions/comments to the new accel sensitivity constant.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@marknolan
marknolan requested a review from jyong15 August 20, 2026 15:52
@marknolan

Copy link
Copy Markdown
Member Author

Correcting the last section of the description: the CSV encoding problem I described does not need a new ticket — it is already fixed and waiting to merge in ShimmerResearch/ASM_PC#365 (DEV-793), which changes all five PrintWriter sites to pass "UTF-8" explicitly. I hadn't spotted that when writing this up.

That also explains the symptom precisely: PrintWriter(File) inherited the platform charset, so the ± in the Sensor config header came out as cp1252 0xB1 on Windows — invalid UTF-8, and byte-different from the same parse on Linux/AWS Lambda where the default is UTF-8.

Nothing changes for this PR. The references regenerated here were produced with JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF-8", so they carry the correct UTF-8 \xc2\xb1 and will still match once #365 lands and the writers set it themselves.

My previous comment claimed ST's accel figures were just 32768/FS rounded to
three significant figures, and that the old derived values were therefore the
more accurate of the two. That was wrong, and the datasheet settles it.

AN5922 Table 27 (FS_XL = +-2 g) gives worked register contents: 1 g reads
0x4009 = 16393 LSB and 350 mg reads 0x1669 = 5737 LSB. 0.061 mg/LSB predicts
16393.4 and 5737.7; a 32768/FS derivation predicts 16384 and 5734.4. The
accelerometer does not map its nominal full scale onto exactly 2^15 counts -
it has a small margin, the same phenomenon as the gyro, just far smaller.

So the old values were wrong by 0.0576%, and this is a correctness fix rather
than a consistency trade-off. The constants themselves are unchanged from the
previous commit; only the reasoning recorded beside them.

Corroborated four ways: the datasheet worked example, ST's upstream driver
(STMicroelectronics/lsm6dsv-pid, byte-identical to the copy this firmware
vendors), every example in STMems_Standard_C_drivers/lsm6dsv_STdC/examples, and
lsm6dsv_self_test.c in particular - its pass/fail limits are specified in mg, so
ST's own self-test only produces correct results if 0.122 mg/LSB is right.

Comment-only; 51 driver tests still pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 20, 2026 19:54
@marknolan

Copy link
Copy Markdown
Member Author

Correction — I had the rationale backwards, and the datasheet settles it. Pushed as aa61da17 (comment-only; the constants are unchanged).

The description above claims ST's accel figures are just 32768/FS rounded to three significant figures, and that the old derived values were therefore the more accurate of the two. That is wrong.

AN5922 Table 27 ("Content of output data registers vs. acceleration", FS_XL = ±2 g) gives worked register contents:

Acceleration Registers LSB 0.061 mg/LSB predicts 32768/FS predicts
1 g 40h 09h 16393 16393.4 16384.0 ✗
350 mg 16h 69h 5737 5737.7 5734.4 ✗
−1 g BFh F7h −16393 −16393.4 −16384.0 ✗

So the accelerometer does not map its nominal full scale onto exactly 2¹⁵ counts — it has a small margin, the same phenomenon as the gyro, just far smaller (0.058% vs ~14.7%). 0.061 mg/LSB is the real sensitivity, not a rounding.

This is therefore a correctness fix, not a consistency trade-off. The old values were wrong by 0.0576%. The outcome is unchanged, but the reasoning recorded in the code now matches reality, so nobody reverts this later on the "32768/FS is exact" argument.

Corroborated four ways:

  1. The datasheet worked example above.
  2. ST's upstream driver STMicroelectronics/lsm6dsv-pid (master) — 0.061f / 0.122f / 0.244f / 0.488f, byte-identical to the copy the firmware already vendors.
  3. Every example in STMems_Standard_C_drivers/lsm6dsv_STdC/examples converts through the lsm6dsv_from_fsN_to_mg helpers; none derives 32768/FS.
  4. lsm6dsv_self_test.c in particular uses from_fs4_to_mg, and self-test pass/fail limits are specified in mg — ST's own self-test only comes out right if 0.122 mg/LSB is the true sensitivity.

The gyro remains untouched and the accel-vs-gyro distinction still holds; it is a difference of degree, not of kind, and the comment now says so.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

ShimmerDriver/src/main/java/com/shimmerresearch/verisense/sensors/SensorLSM6DSV.java:48

  • The class Javadoc sensitivity sentence is grammatically unclear ("0.061 mg/LSB at +-2 g spec of 4.375...") and reads like the accel and gyro clauses are run together. Rewording/punctuation would make the public documentation unambiguous.
 * Sensitivities: the ST datasheet nominal for every axis - 0.061 mg/LSB at +-2 g
 * spec of 4.375 mdps/LSB at +-125 dps for gyro, and 667 LSB/Gauss for the LIS2MDL
 * mag - the last taken from {@link SensorLIS2MDL} rather than duplicated here, so
 * calibrated magnetometer output is in GAUSS, consistent with every other Shimmer
 * magnetometer and with the per-unit calibration the device stores.

ShimmerDriver/src/main/java/com/shimmerresearch/verisense/sensors/SensorLSM6DSV.java:311

  • This accel-sensitivity rationale asserts the 32768/FS derivation is "wrong" and "simply incorrect", which conflicts with the PR description stating this is a consistency change (and that 32768/FS is the more-precise form). Please reconcile the in-code explanation with the PR intent to avoid future readers “fixing” it back based on contradictory rationale.
	// These were previously derived as 32768/(FS_g*9.80665), on the assumption that
	// the accel spans exactly the full 16-bit range at nominal full scale and that
	// ST's printed figures were just that quantity rounded to three significant
	// figures. They are not. The datasheet's own worked example (AN5922 Table 27,
	// FS_XL = +-2 g) settles it: 1 g reads 0x4009 = 16393 LSB and 350 mg reads

Copilot review on PR #288: my earlier edit left the sentence ungrammatical -
dropping "the ST angular-rate" from the first line orphaned the "spec of 4.375
mdps/LSB..." that followed, running the accel and gyro clauses together, and it
listed only the +-2 g accel case.

Now a list giving all three sensors' nominals with their ranges.

Javadoc-only; 51 driver tests still pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 20, 2026 20:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (3)

ShimmerDriver/src/main/java/com/shimmerresearch/verisense/sensors/SensorLSM6DSV.java:328

  • The gyro sensitivity note says "unlike the accel above" even though the preceding accel note explicitly explains the accel also does not map nominal full-scale to exactly 2^15 counts. This contradiction could mislead a future reader into thinking accel is special-cased differently than gyro.
	// gen-1 LSM6DS3. NOTE: unlike the accel above, the gyro genuinely does NOT span
	// the full 16-bit range at its nominal full scale - 4.375 mdps/LSB implies a real
	// full scale of 143.4 dps for the +-125 dps setting, which is exactly where the
	// part is observed to saturate - so here a 32768/FS derivation is not a rounding
	// difference but plainly wrong (~14.7%).

ShimmerDriver/src/test/java/com/shimmerresearch/verisense/API_00008_VerisenseLsm6dsvTaggedFifoParsing.java:154

  • The accel sensitivity literal is repeated across three adjacent assertions, which makes it easy for them to drift if the value is updated again. Defining a local constant keeps the regression-lock literal while reducing duplication.
		assertEquals(-200 / 835.832961457, cal(aligned0, SensorLSM6DSV.ObjectClusterSensorName.LSM6DSV_ACC_X), 0.0001);
		assertEquals(300 / 835.832961457, cal(aligned0, SensorLSM6DSV.ObjectClusterSensorName.LSM6DSV_ACC_Y), 0.0001);
		assertEquals(100 / 835.832961457, cal(aligned0, SensorLSM6DSV.ObjectClusterSensorName.LSM6DSV_ACC_Z), 0.0001);

ShimmerDriver/src/test/java/com/shimmerresearch/verisense/API_00008_VerisenseLsm6dsvTaggedFifoParsing.java:148

  • The comment documents the gyro sensitivity as 57.142857 LSB/dps, but the assertions (and SensorLSM6DSV constants) use 57.142857143. Keeping the comment consistent avoids confusion when debugging test failures.
		// gyro +/-500 dps = 57.142857 LSB/dps (ST 17.50 mdps/LSB);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants