Skip to content

[IDEV-2572] domaintools v14.0: Address Bugbash Feedbacks - #4

Merged
briluza merged 6 commits into
domaintools_v14.0from
IDEV-2572
Aug 26, 2026
Merged

[IDEV-2572] domaintools v14.0: Address Bugbash Feedbacks#4
briluza merged 6 commits into
domaintools_v14.0from
IDEV-2572

Conversation

@briluza

@briluza briluza commented Aug 25, 2026

Copy link
Copy Markdown

Address Bugbash Feedback: enrichment expansion, suspicious flag fixes, and missing domain reporting

Please use a clear and concise title that summarizes your changes.
If this PR is related to an internal Buganizer ticket, please include its ID at the beginning.

  • Added Is Suspicious column to the enrichment table so it's clear which domains
    were flagged alongside their risk category
  • Fixed suspicious flagging to only use the Risk Threshold input as previously
    young domains were always marked suspicious regardless of their score
  • Domains not returned by the API are now reported in the output message and JSON
    result so nothing silently goes missing
  • BulkEnrichDomains now has a Merge With Case Entities option to combine a manual
    domain list with case entities in a single run
  • Expanded the data enriched from the API, both bulk and risk enrichment actions
    now capture significantly more fields including expiration date,
    registrant/registrar details, hosting info, tracking codes, contacts, redirect
    info, and tags
  • GetDomainProfile dates (create date, expiration date) now include the pivot
    count alongside the value
  • Fixed suspicious flag not being cleared on re-run. If a domain previously
    scored above the threshold but drops below it, the flag is now removed instead of
    staying stuck
  • Fixed GetWhoIsHistory checking the wrong API license

Description

Please provide a detailed description of your changes. This helps reviewers understand your work and its context.

What problem does this PR solve?
The DomainTools integration lacked three new enrichment actions (EnrichDomainRisk, BulkEnrichDomains, GetDomainProfile), had several bugs in the existing actions (wrong license check on GetWhoIsHistory, construct_csv imported from the wrong module, suspicious flag never cleared on re-run), and returned limited data from the API as it has missing fields like registrant details, hosting info, tracking codes, contacts, redirect info, and expiration dates.

How does this PR solve the problem?

  • Adds three new actions using iris_enrich and iris_investigate API endpoints with
    full field mapping into typed datamodels
  • Expands EnrichedDomainSummary and IrisInvestigateModel to capture the full API
    response including contacts, tracking codes, tags, hosting, and registration
    fields
  • Fixes is_suspicious to be driven solely by the configurable Risk Threshold input
    and always re-evaluated on each run so stale flags are cleared
  • Adds missing domain reporting across all three actions so domains not returned
    by the API are surfaced in the output instead of silently dropped
  • Adds a Merge With Case Entities parameter to BulkEnrichDomains to support
    combining a manual domain list with in-scope case entities
  • Fixes GetWhoIsHistory license check from parsed-domain-rdap to whois-history
  • Fixes construct_csv import to use TIPCommon.transformation instead of the
    top-level TIPCommon package

Any other relevant information (e.g., design choices, tradeoffs, known issues):

  • Removed widgets.

Checklist:

Please ensure you have completed the following items before submitting your PR.
This helps us review your contribution faster and more efficiently.

General Checks:

  • I have read and followed the project's contributing.md guide.
  • My code follows the project's coding style guidelines.
  • I have performed a self-review of my own code.
  • My changes do not introduce any new warnings.
  • My changes pass all existing tests.
  • I have added new tests where appropriate to cover my changes. (If applicable)
  • I have updated the documentation where necessary (e.g., README, API docs). (If applicable)

Open-Source Specific Checks:

  • My changes do not introduce any Personally Identifiable Information (PII) or sensitive customer data.
  • My changes do not expose any internal-only code examples, configurations, or URLs.
  • All code examples, comments, and messages are generic and suitable for a public repository.
  • I understand that any internal context or sensitive details related to this work are handled separately in internal systems (Buganizer for Google team members).

For Google Team Members and Reviewers Only:

  • I have included the Buganizer ID in the PR title or description (e.g., "Internal Buganizer ID: 123456789" or "Related Buganizer: go/buganizer/123456789").
  • I have ensured that all internal discussions and PII related to this work remain in Buganizer.
  • I have tagged the PR with one or more labels that reflect the pull request purpose.

Screenshots (If Applicable)

N/A


Further Comments / Questions

N/A

@github-actions

Copy link
Copy Markdown

Integration Tests Failed

Click to view the full report

🧩 domain_tools

✅ Passed ❌ Failed ⏭️ Skipped
28 1 0

❌ Failed Tests

tests/test_actions/test_enrich_domain_risk.py::TestEnrichDomainRisk::test_young_domain_marked_suspicious
self = <domain_tools.tests.test_actions.test_enrich_domain_risk.TestEnrichDomainRisk object at 0x7f4b78d598d0>
action_output = MockActionOutput(_out=<_io.StringIO object at 0x7f4b790adb40>, _err=<_io.StringIO object at 0x7f4b78ca97e0>)
dt_manager = <domain_tools.tests.conftest.MockDomainToolsManager object at 0x7f4b78d2bd10>

    @set_metadata(
        integration_config_file_path=CONFIG_PATH,
        entities=[{"identifier": "newdomain.com", "entity_type": "DOMAIN", "additional_properties": {}}],
    )
    def test_young_domain_marked_suspicious(
        self, action_output: MockActionOutput, dt_manager: MockDomainToolsManager
    ):
        dt_manager.set_enrich_domains_response([YOUNG_DOMAIN_RESULT])
    
        EnrichDomainRisk.main()
    
        assert action_output.results.execution_state == ExecutionState.COMPLETED
>       assert "1 marked as suspicious" in action_output.results.output_message
E       AssertionError: assert '1 marked as suspicious' in 'Successfully enriched 1 domain(s). 0 marked as suspicious based on the given threshold of 70. \n'
E        +  where 'Successfully enriched 1 domain(s). 0 marked as suspicious based on the given threshold of 70. \n' = ActionOutput(output_message='Successfully enriched 1 domain(s). 0 marked as suspicious based on the given threshold of...act': None, 'admin_contact': None, 'technical_contact': None, 'billing_contact': None, 'tags': []}}]), debug_output='').output_message
E        +    where ActionOutput(output_message='Successfully enriched 1 domain(s). 0 marked as suspicious based on the given threshold of...act': None, 'admin_contact': None, 'technical_contact': None, 'billing_contact': None, 'tags': []}}]), debug_output='') = MockActionOutput(_out=<_io.StringIO object at 0x7f4b790adb40>, _err=<_io.StringIO object at 0x7f4b78ca97e0>).results

tests/test_actions/test_enrich_domain_risk.py:70: AssertionError

@jbabac jbabac left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. good job @briluza !

@briluza
briluza merged commit ec4e0ef into domaintools_v14.0 Aug 26, 2026
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants