NativeTaskRunner spawns every process with detached: true, so each task is its own process group, and neither the runner nor SparqlAnythingConverter listens for SIGINT or SIGTERM. When the Node process is interrupted, the tasks it started are not: they are reparented to PID 1 and keep running.
Reproduced with sleep 30 through NativeTaskRunner: SIGTERM to node, the child survives with pgid == pid. For geonames-rdf that means Ctrl-C during the 15-minute mapping, or a cancelled CI job, leaves up to concurrency JVMs of -Xmx2g each writing into a sparql-anything-* run directory that convert()’s finally never gets to remove; the next run creates a second one beside it. map.sh has no such gap, since its JVMs are xargs children and die with the script.
Suggestion
Either layer works; the converter is the one that knows what is in flight:
- In
SparqlAnythingConverter.convert(): register a SIGINT/SIGTERM handler for the duration of the run that calls the existing stopInFlight() and removes the run directory, then re-raises the signal. That reuses the abort-with-cleanup path the converter already has for a failed chunk.
- Or in
NativeTaskRunner: track spawned tasks and stop them on process signals, so every consumer gets it.
Related: with detached: true the runner already kills by negative PID in stop(), so the process-group part is in place; what is missing is the trigger.
Found while porting geonames-rdf (netwerk-digitaal-erfgoed/geonames-rdf#50), see #782.
NativeTaskRunnerspawns every process withdetached: true, so each task is its own process group, and neither the runner norSparqlAnythingConverterlistens forSIGINTorSIGTERM. When the Node process is interrupted, the tasks it started are not: they are reparented to PID 1 and keep running.Reproduced with
sleep 30throughNativeTaskRunner:SIGTERMto node, the child survives withpgid == pid. For geonames-rdf that means Ctrl-C during the 15-minute mapping, or a cancelled CI job, leaves up toconcurrencyJVMs of-Xmx2geach writing into asparql-anything-*run directory thatconvert()’sfinallynever gets to remove; the next run creates a second one beside it.map.shhas no such gap, since its JVMs arexargschildren and die with the script.Suggestion
Either layer works; the converter is the one that knows what is in flight:
SparqlAnythingConverter.convert(): register aSIGINT/SIGTERMhandler for the duration of the run that calls the existingstopInFlight()and removes the run directory, then re-raises the signal. That reuses the abort-with-cleanup path the converter already has for a failed chunk.NativeTaskRunner: track spawned tasks and stop them on process signals, so every consumer gets it.Related: with
detached: truethe runner already kills by negative PID instop(), so the process-group part is in place; what is missing is the trigger.Found while porting geonames-rdf (netwerk-digitaal-erfgoed/geonames-rdf#50), see #782.