Skip to content

Validate output log schema independently of metadata version #121

Description

@schmoelder

CADET-RDM currently gates output repository migrations only on the stored cadet_rdm_version.
This misses repositories where .cadet-rdm-data.json already reports the current version, but log.tsv is missing columns required by the current LogEntry schema.

Observed failure:

TypeError: LogEntry.__init__() missing 2 required positional arguments: 'project_repo_branch' and 'project_repo_directory_name'

This happened in a recreated output repository used to purge GitHub-hosted LFS objects.
The output repo metadata was current, but log.tsv had an older or incomplete header.
OutputRepo._update_version() returned early because the metadata version matched the installed CADET-RDM version, so the existing header migration helpers did not run.
Later, ProjectRepo.update_output_main_logs() instantiated OutputLog(self.output_log_file), which parsed the malformed log.tsv and failed before the new entry could be written.

Expected behavior:

CADET-RDM should validate the log.tsv schema independently of the metadata version, at least before parsing it into LogEntry.
If the file exists but misses required columns, CADET-RDM should either:

  1. run the existing header migration helpers;
  2. create a valid empty current log.tsv if there are no usable entries;
  3. or raise a targeted recovery error that explains which columns are missing and how to repair/reset the log.

Useful required-column checks would include at least:

  • output_repo_commit_message
  • output_repo_branch
  • output_repo_commit_hash
  • project_repo_branch
  • project_repo_commit_hash
  • project_repo_directory_name
  • project_repo_remotes
  • python_sys_args
  • tags
  • options_hash

Related design issue:

For output repository reseeding workflows, initialize_output_repo(...) should create a log.tsv state that OutputLog can parse safely, or omit log.tsv entirely until the first run writes it.
The current failure is hard to diagnose because the repository appears current by metadata version, while the log schema is not current.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions