The evaluation is currently running with and for data from commit 3be8d0e.
Back then, the "transformers" field would hold a string, instead of the correct (and expected) list of json objects.
This has been fixed now, but we are in a weird position that evaluation will work for old data, but not for what the main branch produces, and we don't have a full result-set for what the experiments will produce now.
Maybe:
Have a robust extraction method that can work on both?
The evaluation is currently running with and for data from commit 3be8d0e.
Back then, the "transformers" field would hold a string, instead of the correct (and expected) list of json objects.
This has been fixed now, but we are in a weird position that evaluation will work for old data, but not for what the main branch produces, and we don't have a full result-set for what the experiments will produce now.
Maybe:
Have a robust extraction method that can work on both?