Widen sizeInBytes beyond 32-bit Int so files over 2 GB are queryable — Closes #83

Makefile

@@ -47,6 +47,10 @@ api:
 	docker run -d --name api --network cvh-backend-network --network-alias cvh-backend -p 8000:8000 -e SYNC_DATA_DIR=/tmp/sync-data api
 	@echo "API container is up and running on port 8000 (http://0.0.0.0:8000/metadata)."
 
+schema:
+	@echo "Regenerating schema.graphql from the Strawberry schema..."
+	uv run python scripts/export_schema.py
+
 wool:
 	@echo "Building the wool worker Docker image (cfdb-wool, linux/amd64)..."
 	docker build --platform linux/amd64 -t cfdb-wool -f Dockerfile.wool .

README.md

@@ -84,6 +84,7 @@ Run `./certs/generate-certs.sh --help` for full usage information.
 | `make api` | Build and start the API container |
 | `make materialize-files` | Manually materialize all file metadata (usually done via sync) |
 | `make materialize-dcc DCC=hubmap` | Materialize a single DCC |
+| `make schema` | Regenerate the checked-in `schema.graphql` from the Strawberry schema |
 | `make certs` | Generate TLS certificates for production |
 | `make mongodb-prod` | Start MongoDB with TLS/X.509 authentication |
 | `make api-prod` | Start API with X.509 client certificate |
@@ -356,6 +357,31 @@ Single file lookup: `{ file(id: "507f1f77bcf86cd799439011") { filename accessUrl
 
 File count for a filter: `{ fileCount(input: [{ dcc: [{ dccAbbreviation: ["4DN"] }] }]) }` — returns the number of matching files without fetching any documents. It accepts the same `FileMetadataInput` filter shape as `files`; with no `input` it counts every file.
 
+### Custom Scalars
+
+The published contract is `schema.graphql` at the repo root — a generated artifact, regenerated by `make schema` and never edited by hand. Three of its scalars are outside the GraphQL specification, so a typed client must map all three explicitly:
+
+| Scalar | Wire form | Used by |
+|--------|-----------|---------|
+| `ObjectIdScalar` | JSON string | `file(id:)` |
+| `BigInt` | JSON number | `sizeInBytes`, on both `FileMetadataType` and `FileMetadataInput` |
+| `JSON` | Any JSON value | `distinctValues.values`, `collections[].extra.hubmap.metadata` |
+
+`BigInt` is a signed 64-bit integer. The GraphQL specification fixes `Int` at 32 bits, so a file larger than 2,147,483,647 bytes (~2.1 GB) could not be represented at all: the field resolved to `null` and contributed a `Int cannot represent non 32-bit signed integer value` entry to the response's `errors` array, degrading a whole page of results to a partial one. That affects every ENCODE `.hic` file (6–51 GB) and the larger 4DN mcools, so it is the common case for contact maps rather than an edge case. The input filter carries the same scalar — a 32-bit filter would leave exactly the files the widened output field exposes unfilterable. Note that size filtering is exact-match, not a range, and that `size_in_bytes` is stored as a BSON int64 only for ENCODE: 4DN and HuBMAP load it as a string through the C2M2 TSV path, so a numeric equality predicate does not match their documents. That is an ingest-layer gap independent of the scalar's width — the filter was equally inert at `Int` — and normalising it is tracked separately.
+
+`BigInt` stays a JSON **number** on the wire rather than a string, so `sizeInBytes` remains directly usable in client-side arithmetic and comparisons with no parsing step. The usual objection to that choice — values above `Number.MAX_SAFE_INTEGER` (2^53-1) lose precision in JavaScript — does not bind here: 2^53 bytes is ~9 PB, far above any file this API serves. That is a property of *which* fields are routed through the scalar rather than something the scalar enforces, so staying under 2^53 is an admission criterion for any future `BigInt` field. What the scalar does enforce is the signed 64-bit range — exactly where BSON itself stops — plus a rejection of non-integers, including `true`/`false`, on both input and output.
+
+**This is a breaking schema change.** A client that hard-codes `Int` breaks in four ways:
+
+- A query declaring `query Q($s: [Int!])` and passing it to `sizeInBytes` now fails variable-type validation and must declare `[BigInt!]`.
+- Generated clients must re-run codegen against the new SDL **and add a scalar mapping** — `graphql-codegen` and most typed clients silently widen an unrecognised custom scalar to `any`, so `scalars: { BigInt: 'number' }` (or the equivalent) is required or `sizeInBytes` loses its type with no build failure.
+- Any client validating responses against a stored copy of the schema must refresh it.
+- A client sending an integral float — `1234.0`, which any language that round-trips numbers through a float will emit — is now rejected. `Int` coerced it; `BigInt` does not, so the wire form stays one unambiguous representation.
+
+A client that merely *reads* `sizeInBytes` out of the JSON response needs no change — it was already receiving a JSON number, and now receives a correct one instead of `null`.
+
+**Migrating without a window of failures.** The server side of this break is not atomic. A rolling ECS deploy keeps old and new tasks in the target group simultaneously, dev ships on every merge while prod is promoted by hand, and the documented rollback (re-tag an older `:<sha>`) reverts the schema under clients that have already migrated. In each case a client that *names* `BigInt` in a variable declaration fails against whichever tasks are still on the old schema. The way out is that the leaf type only has to be named when the client declares a variable for it: passing the filter as an inline literal (`sizeInBytes: [6262125716]`), or hoisting the variable up to the whole input object (`query Q($input: [FileMetadataInput!])`), validates against the old schema and the new one alike. Move consumers to one of those forms first, and the deploy — in either direction — is a non-event.
+
 ### Query Mechanics
 
 The GraphQL API uses an implicit OR/AND clause system for building MongoDB queries:
@@ -460,7 +486,7 @@ The central entity representing a stable digital asset.
 | `dcc` | Dcc | The Data Coordinating Center that produced this file |
 | `collections` | Collection[] | Collections containing this file |
 | `project` | Project? | The primary project within which this file was created |
-| `size_in_bytes` | int? | File size |
+| `size_in_bytes` | int? | File size, exposed as the 64-bit `BigInt` scalar (see [Custom Scalars](#custom-scalars)) |
 | `sha256` | string? | SHA-256 checksum (preferred) |
 | `md5` | string? | MD5 checksum (if SHA-256 unavailable) |
 | `filename` | string | Filename without path |

schema.graphql

@@ -22,6 +22,11 @@ type AssayTypeType {
   description: String
 }
 
+"""
+A signed 64-bit integer, serialized as a JSON number. Widens the GraphQL `Int` scalar, which the specification fixes at 32 bits and which therefore cannot represent a file larger than ~2 GB. Values beyond 9007199254740991 (2^53-1) exceed what an IEEE-754 double represents exactly and would lose precision in a JavaScript client; byte sizes cannot reach that magnitude.
+"""
+scalar BigInt
+
 input BiosampleInput {
   idNamespace: [String!] = null
   localId: [String!] = null
@@ -407,7 +412,7 @@ input FileMetadataInput {
   projectLocalId: [String!] = null
   persistentId: [String!] = null
   creationTime: [String!] = null
-  sizeInBytes: [Int!] = null
+  sizeInBytes: [BigInt!] = null
   sha256: [String!] = null
   md5: [String!] = null
   filename: [String!] = null
@@ -443,7 +448,7 @@ type FileMetadataType {
   projectLocalId: String!
   persistentId: String
   creationTime: String
-  sizeInBytes: Int
+  sizeInBytes: BigInt
   sha256: String
   md5: String
   filename: String!
@@ -555,4 +560,4 @@ type SubjectType {
   race: [String!]!
   taxonomy: NcbiTaxonomyType
   extra: EnrichedSubjectType
-}
+}

scripts/export_schema.py

@@ -0,0 +1,32 @@
+"""Regenerate the checked-in GraphQL SDL from the Strawberry schema.
+
+``schema.graphql`` is a generated artifact — edit the Python types and run
+this script (``make schema``) rather than editing the SDL by hand.
+``tests/test_schema.py`` fails when the two drift apart.
+"""
+
+from pathlib import Path
+
+from strawberry.printer import print_schema
+
+from cfdb.api.gql.schema import schema
+
+
+SCHEMA_PATH = Path(__file__).resolve().parent.parent / "schema.graphql"
+
+
+def render() -> str:
+    """Return the SDL exactly as it is written to ``schema.graphql``."""
+    return print_schema(schema) + "\n"
+
+
+def main() -> None:
+    # Explicit encoding, not the process locale: the artifact's bytes are
+    # compared for equality by the drift test, so they must not depend on
+    # the shell of whoever ran ``make schema``.
+    SCHEMA_PATH.write_text(render(), encoding="utf-8")
+    print(f"Wrote {SCHEMA_PATH}")
+
+
+if __name__ == "__main__":
+    main()

src/cfdb/api/gql/inputs.py

@@ -2,6 +2,8 @@
 
 import strawberry
 
+from cfdb.api.gql.types import BigInt
+
 
 @strawberry.input
 class AnatomyInput:
@@ -243,7 +245,10 @@ class FileMetadataInput:
     project_local_id: list[str] | None = None
     persistent_id: list[str] | None = None
     creation_time: list[str] | None = None
-    size_in_bytes: list[int] | None = None
+    # ``BigInt``, not ``int``: a 32-bit ``Int`` filter cannot name the size of
+    # a file over ~2 GB, which would leave exactly the files the widened
+    # output field exposes unfilterable.
+    size_in_bytes: list[BigInt] | None = None
     sha256: list[str] | None = None
     md5: list[str] | None = None
     filename: list[str] | None = None

src/cfdb/api/gql/types.py

@@ -3,6 +3,7 @@
 import strawberry
 import strawberry.scalars
 from bson import ObjectId
+from graphql import GraphQLError
 from pydantic import BaseModel
 
 from cfdb.models import FileMetadataModel
@@ -18,6 +19,73 @@ class ObjectIdScalar:
     ...
 
 
+# Bounds of a signed 64-bit integer — the range ``BigInt`` accepts, matching
+# what MongoDB stores in a BSON int64 and what a C2M2 file size can be.
+_INT64_MIN = -(2**63)
+_INT64_MAX = 2**63 - 1
+
+# Above this magnitude a JSON number is no longer exactly representable in an
+# IEEE-754 double, which is the only numeric type a browser client has. This
+# is surfaced in the scalar's description and is deliberately NOT enforced:
+# the 64-bit bound above is the one every downstream layer shares (it is
+# exactly where BSON raises), so clamping here would invent a second, softer
+# limit inside a type whose name promises 64 bits. Staying JS-safe is
+# therefore an admission criterion for routing a field through ``BigInt`` —
+# ``size_in_bytes`` qualifies because 2**53 bytes is ~9 PB — rather than
+# something the scalar checks.
+_JS_SAFE_INTEGER_MAX = 2**53 - 1
+
+
+def _coerce_big_int(value):
+    """Coerce a ``BigInt`` value in either direction, rejecting non-integers.
+
+    Serialization and parsing share one implementation because the wire form
+    is a JSON number: the value that goes out is the value that comes back.
+    ``bool`` is excluded explicitly because it is an ``int`` subclass in
+    Python and ``true`` is not a size. Integral floats are rejected too, so
+    the wire form stays one unambiguous representation — note this is
+    stricter than the ``Int`` it replaces, which coerced them.
+
+    Raises ``GraphQLError`` rather than ``ValueError`` because graphql-core
+    logs a non-``GraphQLError`` cause with its traceback: on an
+    unauthenticated endpoint every malformed filter value would otherwise
+    write a stack trace at ERROR.
+    """
+    if isinstance(value, bool) or not isinstance(value, int):
+        raise GraphQLError(f"BigInt cannot represent non-integer value: {value!r}")
+    if not _INT64_MIN <= value <= _INT64_MAX:
+        raise GraphQLError(
+            f"BigInt cannot represent non 64-bit signed integer value: {value}"
+        )
+    return value
+
+
+@strawberry.scalar(
+    description=(
+        "A signed 64-bit integer, serialized as a JSON number. Widens the "
+        "GraphQL `Int` scalar, which the specification fixes at 32 bits and "
+        "which therefore cannot represent a file larger than ~2 GB. Values "
+        f"beyond {_JS_SAFE_INTEGER_MAX} (2^53-1) exceed what an IEEE-754 "
+        "double represents exactly and would lose precision in a JavaScript "
+        "client; byte sizes cannot reach that magnitude."
+    ),
+    serialize=_coerce_big_int,
+    parse_value=_coerce_big_int,
+)
+class BigInt:
+    """A signed 64-bit integer represented as a JSON number in GraphQL."""
+
+    ...
+
+
+# Model fields whose Python ``int`` annotation must NOT become a GraphQL
+# ``Int``. Keyed by ``(model, field name)`` so widening one model's field
+# does not silently widen an unrelated field that happens to share its name.
+_SCALAR_OVERRIDES: dict[tuple[type, str], object] = {
+    (FileMetadataModel, "size_in_bytes"): BigInt,
+}
+
+
 @strawberry.type
 class DistinctFieldType:
     field: str
@@ -104,6 +172,28 @@ def _resolve_json_type(field_type):
     return field_type
 
 
+def _substitute_scalar(field_type, scalar):
+    """Replace a field's scalar type, preserving an ``Optional`` wrapper.
+
+    Only ``T`` and ``Optional[T]`` are handled, and anything else raises
+    rather than being approximated. A silently mishandled wrapper would
+    publish a structurally wrong SDL — a list field emitted as a bare
+    scalar — that the drift test would then bless on the next
+    ``make schema``, so an unsupported shape has to fail here, at import.
+    """
+    args = getattr(field_type, "__args__", None)
+    if args is None:
+        return scalar
+    inner = [a for a in args if a is not type(None)]
+    # ``Optional[List[T]]`` also has two args, one of them ``None`` — hence
+    # the check that what remains is itself unparameterized.
+    if len(args) == 2 and len(inner) == 1 and not hasattr(inner[0], "__args__"):
+        return Optional[scalar]
+    raise TypeError(
+        f"scalar override supports only T and Optional[T], not {field_type!r}"
+    )
+
+
 def _rebuild_type(field_type, model_cls, strawberry_cls):
     """Replace a BaseModel class inside a (possibly nested) type annotation
     with its Strawberry equivalent, preserving Optional/List wrappers."""
@@ -146,6 +236,12 @@ def wrapper(cls):
         if name:
             cls.__name__ = f"{name}Type"
         for field_name, field_type in get_type_hints(model).items():
+            override = _SCALAR_OVERRIDES.get((model, field_name))
+            if override is not None:
+                cls.__annotations__[field_name] = _substitute_scalar(
+                    field_type, override
+                )
+                continue
             model_cls, _ = _find_basemodel(field_type)
             if model_cls is None:
                 if field_type is ObjectId:

tests/conftest.py

@@ -9,6 +9,12 @@
 from cfdb import api
 
 
+# The exact size, in bytes, of the ENCODE .hic file named in issue #83 —
+# above the 2**31-1 ceiling GraphQL's ``Int`` scalar imposes. Shared so the
+# number that names the regression exists in exactly one place.
+ISSUE_83_SIZE = 6262125716
+
+
 def _resolve(doc: dict, key: str):
     """Resolve a possibly dot-notated key against a nested dict."""
     value = doc