Skip to content

fix(dart): use BigInt for proper dart web truncation#3600

Open
ayush00git wants to merge 6 commits intoapache:mainfrom
ayush00git:fix/writeVarI64-zigzag
Open

fix(dart): use BigInt for proper dart web truncation#3600
ayush00git wants to merge 6 commits intoapache:mainfrom
ayush00git:fix/writeVarI64-zigzag

Conversation

@ayush00git
Copy link
Copy Markdown
Contributor

@ayush00git ayush00git commented Apr 21, 2026

Why?

In the web fallback path (_useBigIntVarint64 == true), the code calculates the Zig-Zag encoding mask using BigInt.from(value >> 63). On Dart Web, standard bitwise operators (>>) truncate 64-bit Dart integers to 32 bits before shifting. Therefore, if a positive 64-bit number (fitting within the 53-bit safe integer range) has the 31st bit set (e.g., 3,000,000,000), it is treated as a negative 32-bit integer. value >> 63 evaluates to -1 instead of 0. This XORs the signed << 1 value with all 1s, completely inverting the zig-zag value and encoding a positive number as a negative varint.

What does this PR do?

Use the BigInt shift operator instead, which does not truncate, or a simple zero-check.

Related issues

AI Contribution Checklist

  • Substantial AI assistance was used in this PR: yes / no
  • If yes, I included a completed AI Contribution Checklist in this PR description and the required AI Usage Disclosure.
  • If yes, my PR description includes the required ai_review summary and screenshot evidence of the final clean AI review results from both fresh reviewers on the current PR diff or current HEAD after the latest code changes.

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?
  • Does this PR introduce any binary protocol compatibility change?

Benchmark

@chaokunyang
Copy link
Copy Markdown
Collaborator

Could you add a test to cover it, and is there any code example to reproduce it?

@ayush00git
Copy link
Copy Markdown
Contributor Author

Could you add a test to cover it, and is there any code example to reproduce it?

@chaokunyang this may reproduce the bug and also show the verification of the fix

void main() {
  const value = 3000000000; // positive, fits in JS safe-integer range
  // Old (buggy) web fallback — value >> 63 is evaluated as a 32-bit int:
  //   3000000000 as int32 == -1294967296  (bit 31 set → negative)
  //   -1294967296 >> 63   == -1           (fills with sign bit)
  final buggyZigZag = (BigInt.from(value) << 1) ^ BigInt.from(value >> 63);
  print(buggyZigZag); // Web: large negative number — CORRUPT
                       // Native: 6000000000 — correct (native >> is 64-bit)
  // Fixed path — shift the BigInt, no truncation:
  final signed = BigInt.from(value);
  final fixedZigZag = (signed << 1) ^ (signed >> 63);
  print(fixedZigZag); // Always: 6000000000 — correct on both platforms
}

@chaokunyang
Copy link
Copy Markdown
Collaborator

The tests won't run actually, coudl you add dart test -p chrome to ci.yaml?

@ayush00git
Copy link
Copy Markdown
Contributor Author

The tests won't run actually, coudl you add dart test -p chrome to ci.yaml?

my bad, let me update ci.yml

@ayush00git
Copy link
Copy Markdown
Contributor Author

@chaokunyang the new ci script exposed more failing tests in the packages/fory/test file. dart2js compiles the entire file. the only reliable solution is to split the web-unsafe tests into a separate file, or annotate each test body to skip the literals on web.
The cleanest path would be to put the web regression test in its own separate file. should I do that ?

@chaokunyang
Copy link
Copy Markdown
Collaborator

@chaokunyang the new ci script exposed more failing tests in the packages/fory/test file. dart2js compiles the entire file. the only reliable solution is to split the web-unsafe tests into a separate file, or annotate each test body to skip the literals on web. The cleanest path would be to put the web regression test in its own separate file. should I do that ?

No, that still doesn't fory dart on web work. It's just avoid real issue.

@ayush00git
Copy link
Copy Markdown
Contributor Author

No, that still doesn't fory dart on web work. It's just avoid real issue.

@chaokunyang making dart test -p chrome fully pass on buffer_test.dart requires a larger fix that is adding BigInt-based fallbacks to the raw 64-bit read/write methods in buffer.dart and generated_support.dart, plus rewriting the affected test literals.
Should I keep the scope of current PR only for the writeVarInt64 fix, and for this bug open an issue and then start working on it in a new PR ?

@ayush00git
Copy link
Copy Markdown
Contributor Author

for now we could add

@TestOn('vm')
library;

for just passing the ci checks ? or should I fix everything in this PR only ?

@chaokunyang
Copy link
Copy Markdown
Collaborator

Need a full fix, but that's a big work, and break existing fory dart object model.

@ayush00git
Copy link
Copy Markdown
Contributor Author

Need a full fix, but that's a big work, and break existing fory dart object model.

so how should I proceed ? Should I propose a proper fix in a github issue first ? and then begin with the PR?

@chaokunyang
Copy link
Copy Markdown
Collaborator

chaokunyang commented Apr 22, 2026

Such fix is not useful, the dart web support is a big work. Either support it fully, or don't support it.

The dart int will be compiled to javascript numer, which is double, int64 can't be expressed using double. and if we use bigint, it's too slow and we paid cost for dart native ,which is unnecessary.

So, we should not add any half dart web fix until we have a good design that can address both issues

@ayush00git
Copy link
Copy Markdown
Contributor Author

Such fix is not useful, the dart web support is a big work. Either support it fully, or don't support it.

The dart int will be compiled to javascript numer, which is double, int64 can't be expressed using double. and if we use bigint, it's too slow and we paid cost for dart native ,which is unnecessary.

So, we should not add any half dart web fix until we have a good design that can address both issues

fair enough, so let's keep dart web support as a future feature and you can close this PR then. as i'm free i'll look into a design which could address both these issues.

@ayush00git
Copy link
Copy Markdown
Contributor Author

@chaokunyang we surely need a good design, the float16.double and bfloat16.double methods crashes as well.

UnsupportedError: Uint64 accessor not supported by dart2js.

do we have any plans to implement dart web in future?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants