SparqlAnythingConverter.convert() rejects any process whose --output is empty, on the reasoning that SPARQL Anything exits 0 when it cannot read or parse an input. That reasoning is right, but the check cannot tell that case from a chunk whose rows were all filtered out by the query, which is legitimate and also yields a 0-byte file.
Reproduction
A chunk of three alternateNamesV2 rows whose isolanguage is link, post and fr_1793, run through geonames-rdf’s config/alternate-names.rq, which filters all three out. The pinned jar (1.2.0-RC1) writes a 0-byte --output and exits 0. convert() then throws:
SPARQL Anything produced no output for ‘config/alternate-names.rq’ over ‘…’; it exits successfully when it cannot read or parse an input
and stops the chunks still running. map.sh in geonames-rdf accepts such a chunk. With the default 1M rows a remainder chunk this selective is unlikely, but a small chunk size for a smoke test hits it, and the message then sends the reader after the wrong cause.
Suggested fix
Distinguish “no triples” from “unreadable input”. Options, roughly in order of preference:
- Only treat an empty output as a failure when the chunk itself could not have produced one: check that the chunk (or
load) file exists and is non-empty before starting the process, and accept an empty output otherwise. That keeps the guard for the case it was written for – a missing or unparsable file – and lets an all-filtered chunk through.
- Read SPARQL Anything’s output for the log line it writes when it cannot read an input, and fail on that rather than on the size of the result.
- Least good: an opt-out per job.
Found while porting geonames-rdf (netwerk-digitaal-erfgoed/geonames-rdf#50), see #782.
SparqlAnythingConverter.convert()rejects any process whose--outputis empty, on the reasoning that SPARQL Anything exits 0 when it cannot read or parse an input. That reasoning is right, but the check cannot tell that case from a chunk whose rows were all filtered out by the query, which is legitimate and also yields a 0-byte file.Reproduction
A chunk of three
alternateNamesV2rows whoseisolanguageislink,postandfr_1793, run through geonames-rdf’sconfig/alternate-names.rq, which filters all three out. The pinned jar (1.2.0-RC1) writes a 0-byte--outputand exits 0.convert()then throws:and stops the chunks still running.
map.shin geonames-rdf accepts such a chunk. With the default 1M rows a remainder chunk this selective is unlikely, but a small chunk size for a smoke test hits it, and the message then sends the reader after the wrong cause.Suggested fix
Distinguish “no triples” from “unreadable input”. Options, roughly in order of preference:
load) file exists and is non-empty before starting the process, and accept an empty output otherwise. That keeps the guard for the case it was written for – a missing or unparsable file – and lets an all-filtered chunk through.Found while porting geonames-rdf (netwerk-digitaal-erfgoed/geonames-rdf#50), see #782.