task-runner: expose the runner’s capacity so the converter can size its pool

[ddeboer]

SparqlAnythingConverter defaults concurrency to 1, on the reasoning that it cannot see the machine its task runner spawns on. That is true of the converter, but not of the task runner: NativeTaskRunner spawns on the host it runs on, and DockerTaskRunner knows the container it starts. Neither exposes what it knows, because TaskRunner is only run/wait/stop.

The consequence shows in the first consumer. geonames-rdf’s lde/map.ts carries 25 lines that copy map.sh’s pool sizing – CPU count, cgroup v2 and v1 memory limits, os.totalmem() as the fallback, a 3 GB budget per worker – so that four -Xmx2g JVMs fit a 16 GB runner and sixteen do not get launched on a many-core pod with a small memory limit. Nothing in that block is GeoNames policy: the budget is the heap plus fixed headroom. The next consumer either copies it, and re-applies every fix to it, or runs at 1×.

Suggestion

A capacity() on the task runner:

interface TaskRunner<Task> {
  run(command: string): Promise<Task>;
  wait(task: Task): Promise<string>;
  stop(task: Task): Promise<string | null>;
  capacity?(): Promise<{ cpus: number; memoryBytes: number }>;
}

• NativeTaskRunner: os.availableParallelism() and the cgroup limit when there is one, else os.totalmem().
• DockerTaskRunner: the container’s --cpus/--memory when set, else the daemon’s.

The converter then derives its default: min(cpus, floor(memoryBytes / (heap + headroom))), with concurrency still overriding it. lde/map.ts keeps only the PARALLELISM override, and the README row that says the converter cannot see the machine goes.

Optional on the interface, so a remote runner that cannot answer keeps the default of 1.

Found while porting geonames-rdf (netwerk-digitaal-erfgoed/geonames-rdf#50), see #782.