When an arff file contains an unknown value ? with an attribute of type numeric then it is transformed into NaN in the data. This is expected behavior.
However, when normalizing, these values are not ignored and thus an entire column will get set to NaN due to a single unknown.
The root of the problem is in the min and max functions, which end up returning NaN, throwing off the normalize function.
When an arff file contains an unknown value
?with an attribute of typenumericthen it is transformed intoNaNin the data. This is expected behavior.However, when normalizing, these values are not ignored and thus an entire column will get set to
NaNdue to a single unknown.The root of the problem is in the min and max functions, which end up returning
NaN, throwing off the normalize function.