Add SourceConceptId uniqueness validation to validateUsagiFile

[Copilot]

validateUsagiFile did not detect cases where the same ADD_INFO:sourceConceptId was assigned to multiple distinct source codes — a constraint violation since each source concept must own a unique concept ID.

Changes

• R/validateUsagiFile.R — Added uniqueness check in the C&CR validation block using the validate:: package. Deduplicates to distinct (sourceCode, sourceConceptId) pairs first (to avoid false positives from valid multi-mapped codes where one source code legitimately maps to multiple target concepts), then uses is_unique() to detect sourceConceptIds shared across different source codes:

  usagiDistinct <- usagiTibble |>
      dplyr::filter(!is.na(`ADD_INFO:sourceConceptId`)) |>
      dplyr::distinct(sourceCode, `ADD_INFO:sourceConceptId`)
  
  distinctValidationRules <- validate::validator(
      SourceConceptId.is.not.unique = is_unique(`ADD_INFO:sourceConceptId`)
  )
  distinctValidations <- validate::confront(usagiDistinct, distinctValidationRules)

  Affected rows are flagged with "ERROR: SourceConceptId is not unique".

• inst/testdata/VOCABULARIES/ICD10fi/ICD10fi_with_errors.usagi.csv

  • Fixed pre-existing data issue: rows A17.8+G63.1 and A17.8+G63.2 were unintentionally sharing sourceConceptId=2000500119 with A17.8+G63.0; assigned unique IDs 2000599997/2000599998.
  • Added row A99.0 ([SourceConceptId is not unique]Test duplicate sourceConceptId) that reuses sourceConceptId=2000500124 (also used by A18.0+M68.0) to deliberately trigger the new rule.

• tests/testthat/test-validateUsagiFile.R — Added assertions verifying the new validation step appears in the summary, flags exactly the intended row, sets the correct ADD_INFO:validationMessages, and sets mappingStatus to FLAGGED.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Usagi validation is not detecting duplicated source concept ids</issue_title>
<issue_description>SourceConceptId must be unique
this means that in the usagi file
SourceConceptId + SourceCode must be unique
this has to be added to the rules in the validate usagi function
also a test must be added to corresponding file test
and a test row to the usagi with error ICD10 file in order to trigger the test </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Usagi validation is not detecting duplicated source concept ids #69

---

⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.

[javier-gracia-tabuenca-tuni]

why dont you do this in the upper section using the validate:: package
just like
SourceConceptId.SouceCodes.are.unique = is_unique(ADD_INFO:sourceConceptId, sourceCode)

[Copilot]

Refactored in commit 3517032 to use validate:: — however, is_unique(sourceConceptId, sourceCode) directly on the full tibble would incorrectly flag valid multi-mapped codes (the valid file has 420 source codes that appear multiple times with the same sourceConceptId but different conceptIds).

The fix deduplicates to distinct (sourceCode, sourceConceptId) pairs first, then uses is_unique(sourceConceptId) on that deduplicated tibble:

usagiDistinct <- usagiTibble |>
    dplyr::filter(!is.na(`ADD_INFO:sourceConceptId`)) |>
    dplyr::distinct(sourceCode, `ADD_INFO:sourceConceptId`)

distinctValidationRules <- validate::validator(
    SourceConceptId.is.not.unique = is_unique(`ADD_INFO:sourceConceptId`)
)
distinctValidations <- validate::confront(usagiDistinct, distinctValidationRules)

[javier-gracia-tabuenca-tuni]

@copilot can you look at the error from the actions and fix it

[Copilot]

@javier-gracia-tabuenca-tuni The model claude-sonnet-4.6 is not available for your account. This can happen if the model was disabled by your organization's policy or if your Copilot plan doesn't include access to it.

You can try again without specifying a model (just @copilot) to use the default, or choose a different model from the model picker.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 1c05da9f-f863-44c5-ad7d-4c5f6186af2a

[Copilot]

@javier-gracia-tabuenca-tuni The model claude-sonnet-4.6 is not available for your account. This can happen if the model was disabled by your organization's policy or if your Copilot plan doesn't include access to it.

You can try again without specifying a model (just @copilot) to use the default, or choose a different model from the model picker.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 1c05da9f-f863-44c5-ad7d-4c5f6186af2a

[Copilot]

@javier-gracia-tabuenca-tuni The model claude-sonnet-4.6 is not available for your account. This can happen if the model was disabled by your organization's policy or if your Copilot plan doesn't include access to it.

You can try again without specifying a model (just @copilot) to use the default, or choose a different model from the model picker.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 1c05da9f-f863-44c5-ad7d-4c5f6186af2a

[Copilot]

@javier-gracia-tabuenca-tuni The model claude-sonnet-4.6 is not available for your account. This can happen if the model was disabled by your organization's policy or if your Copilot plan doesn't include access to it.

You can try again without specifying a model (just @copilot) to use the default, or choose a different model from the model picker.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 1c05da9f-f863-44c5-ad7d-4c5f6186af2a

[Copilot]

@javier-gracia-tabuenca-tuni The model claude-sonnet-4.6 is not available for your account. This can happen if the model was disabled by your organization's policy or if your Copilot plan doesn't include access to it.

You can try again without specifying a model (just @copilot) to use the default, or choose a different model from the model picker.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 1c05da9f-f863-44c5-ad7d-4c5f6186af2a

[javier-gracia-tabuenca-tuni]

@copilot

actions return this error

|
|======================================================================| 100%
Executing SQL took 0.00365 secs
Inserting data took 0.0109 secs
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 248 ]

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-validateUsagiFile.R:196:3'): test validateUsagiFile returns errors with the errored usagi file ──
Expected dplyr::pull(...) to equal "ERROR: SourceConceptId is not unique".
Differences:
actual vs expected

• "ERROR OUTDATED conceptName: conceptName for conceptId 4121541 is different in the target vocabularies | ERROR: SourceConceptId is not unique | ERROR: A99.0 is not a valid concept code in vocabulary ICD10"

• "ERROR: SourceConceptId is not unique"

fix it

[Copilot]

@copilot

actions return this error
...

Fixed in commit 3df863e. The A99.0 row was triggering two extra validation errors:

1. OUTDATED conceptName — conceptId 4121541 is used by another row (A17.8+G63.1) with the wrong name "Neutropenia", and the outdated-concepts join propagates that error to all rows with the same conceptId, including A99.0.
2. Invalid parent concept code — A99.0 doesn't exist in the ICD10 vocabulary in the test DB.

Replaced A99.0 with A18.8+E35.0 which uses:

• A valid ICD10fi source code with parent codes (A18|A18.8|E35.0) all present in the test DB
• A clean conceptId 141777 ("Tuberculosis of thyroid gland") not used anywhere else in the error file, matching the test DB name exactly