Skip to content

feat(18): drop the local utils.py - #26

Merged
tdudgeon merged 2 commits into
mainfrom
feat/18-drop-local-utils
Aug 11, 2026
Merged

feat(18): drop the local utils.py#26
tdudgeon merged 2 commits into
mainfrom
feat/18-drop-local-utils

Conversation

@tdudgeon

Copy link
Copy Markdown
Collaborator

Second and final PR for this repo under InformaticsMatters/squonk2-jobs#18,
after #24 and #25 which adopted the shared CLI helpers.

Summary

The 35 modules importing the local utils now take it from
dm_job_utilities, which provides all of log, expand_path,
read_delimiter, is_type and calc_geometric_mean. Importing the module
rather than the names — from dm_job_utilities import utils — leaves every
utils.x call site unchanged, so the diff is one line per file.

utils.py (157 lines) is deleted. Net −206/+109.

The two #18 blockers

This repo is the only consumer that actually reached them.

round_to_significant_numbersigfig.round

5 uses, all in dmpk/pk_tmax_cmax_sim.py (Tmax, Cmax, Kel, Ka, V/F at 3 s.f.).

The local implementation leaned on builtin round() and inherited its half-way
behaviour — job-utilities' README calls it inaccurate, and that checks out:

value local sigfig
2.675 2.67 2.68
1.005 1.0 1.01

Over 12,000 randomly generated PK-plausible values, none differ. The
divergence needs a value sitting exactly on a decimal boundary, which typed
constants produce and simulation output effectively never does. So this is a
correctness fix with no expected change to real output — but it is a numeric
change to a pharmacokinetics Job, which is why it was kept out of #24.

Wrapped locally as sigfig_round() to pass warn=False: sigfig warns when a
value carries fewer significant figures than requested (e.g. 7.0 at 3 s.f.),
which is common here and was silent before. Without it the Job log would fill
with UserWarning.

get_path_from_digestdigest_utils.py

Used by two scripts, assemble_conformers.py and
prepare_enum_conf_lists.py. job-utilities dropped it as obsolete, but the
sharded directory layout it describes is still what one script writes and the
other reads back — it is not obsolete here. Moved to its own small module rather
than inlined into both. Verified identical to the original across 2,020 cases
(500 real digests × 4 chars/levels configurations, plus degenerate inputs).

Dockerfiles

Every Dockerfile installing job-utilities is bumped to >= 1.4.0.

This was load-bearing, not tidying: dm_job_utilities.utils does not exist
before 1.3.0 — I checked the published wheels. Dockerfile-dmpk and
Dockerfile-rdock were pinned at 1.0.1, and Dockerfile-fns and
Dockerfile-plants at 1.1.1, so all four would have failed at import.

sigfig is added to Dockerfile-dmpk, which did not carry it (Dockerfile-prep
already did, for sa_score.py).

Verification

Every utils.* reference in the repo was resolved against the shared module
programmatically — no unresolved names.

Output parity against main, same inputs:

Script Result
rdkit_props.py byte-identical
rdkit_dedup.py byte-identical
sa_score.py byte-identical
cluster_butina.py byte-identical
le_conformers.py byte-identical

max_min_picker.py differs — but it differs against itself run twice on
main. It's non-deterministic, so unrelated to this change.

assemble_conformers.py and prepare_enum_conf_lists.py (the digest consumers)
import cleanly. dmpk/pk_tmax_cmax_sim.py can't be imported locally
(matplotlib absent — identical on main), so its changed rounding function was
exercised directly.

All 36 touched Python files parse.

jote was not run — needs the affected images rebuilt. Worth a CI run,
particularly for the dmpk and rdock images whose pins moved furthest.

🤖 Generated with Claude Code

tdudgeon and others added 2 commits August 11, 2026 14:16
The 35 modules that imported the local utils now take it from
dm_job_utilities, which provides all of log, expand_path, read_delimiter,
is_type and calc_geometric_mean. Importing the module rather than the names
(`from dm_job_utilities import utils`) leaves every utils.x call site
unchanged.

This repo is the only consumer that actually reached the two functions #18
flags as blockers:

- round_to_significant_number (5 uses, all in dmpk/pk_tmax_cmax_sim.py) is
  replaced by sigfig.round, as job-utilities' README directs. The local
  implementation leaned on builtin round() and so inherited its half-way
  behaviour: 2.675 rounded to 2.67 and 1.005 to 1.0, where sigfig gives 2.68
  and 1.01. Over 12000 randomly generated PK-plausible values, none differ -
  simulated values do not land on exact decimal boundaries - so this is a
  correctness fix with no expected change to real output. Wrapped locally to
  pass warn=False, since sigfig warns when a value carries fewer significant
  figures than requested and the previous implementation was silent.

- get_path_from_digest (used by assemble_conformers.py and
  prepare_enum_conf_lists.py) was deliberately dropped from job-utilities as
  obsolete, but the sharded directory layout it describes is still what one
  script writes and the other reads. It moves to digest_utils.py rather than
  being inlined into both.

Every Dockerfile installing job-utilities is bumped to >= 1.4.0:
dm_job_utilities.utils first appears in 1.3.0, so the images pinned at 1.0.1
and 1.1.1 would have failed at import. sigfig is added to Dockerfile-dmpk,
which did not carry it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three separate faults, all mine, all in the Dockerfiles:

- Five images COPY utils.py explicitly (fns, dmpk, plants, jaqpot, mordred);
  with the file gone the COPY could not resolve. Removed. The images that
  glob (COPY *.py) were unaffected, and that glob is also how digest_utils.py
  reaches vs-prep, which is where both its consumers run.

- Changing the pins from == to >= left them unquoted, so /bin/sh read the >
  as a redirection: `pip install foo>=1.4.0` installed foo unpinned and wrote
  stdout to a file named "=1.4.0". Every spec containing >= or <= is now
  quoted. The builds that passed had been silently installing unpinned.

- Dockerfile-rdock never needed bumping: prepare_rdock.py imports no utils at
  all, and the image is Python 2 (rdock:2013.1), where a modern
  job-utilities cannot install. Reverted to its original pin.

Also bumps the remaining im-rdkit-utilities==1.0.0 pins to >= 1.1.2. That
version pins job-utilities to ==1.3.0, so it conflicts with >= 1.4.0 - the
same conflict rdkit-utilities 1.1.2 was released to fix. It only surfaced
once the quoting was corrected.

All nine images build locally, resolve dm_job_utilities.utils (and
rdkit_utils where used) at the intended versions, and carry no stray
redirect file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tdudgeon
tdudgeon merged commit a08154e into main Aug 11, 2026
12 checks passed
@tdudgeon
tdudgeon deleted the feat/18-drop-local-utils branch August 11, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant