Skip to content

[FIX] Patch the lookups bridge in the profile-fallback tests (fixes cloud CI) - #2227

Closed
pk-zipstack wants to merge 1 commit into
mainfrom
fix/profile-fallback-test-cloud-overlay
Closed

[FIX] Patch the lookups bridge in the profile-fallback tests (fixes cloud CI)#2227
pk-zipstack wants to merge 1 commit into
mainfrom
fix/profile-fallback-test-cloud-overlay

Conversation

@pk-zipstack

Copy link
Copy Markdown
Contributor

What

Adds the missing get_lookup_config patch to the collaborator list in test_profile_resolution_fallback.py (added in #2203).

Why

The suite calls the real build_fetch_response_payload and patches every collaborator — except the lookups bridge. In OSS that bridge is a no-op, so the suite is green here. But cloud CI runs this same suite against the merged tree, where prompt_studio/lookup_utils.get_lookup_config resolves to the real pluggable_apps/lookups/execution.py implementation — which feeds the test's MagicMock prompt into an ORM query. Since 892be32 landed, every cloud CI run fails four of these tests with:

django.core.exceptions.ValidationError: ['“[]” is not a valid UUID.']

First failing cloud-main run: 31078494222 (cloud main was green at 06:01, red at 06:47 — bracketing #2203's merge). The same failures currently block unrelated cloud PRs (e.g. Zipstack/unstract-cloud#1690).

How

One line + comment: patch.object(psh, "get_lookup_config", autospec=True, return_value=None) — returning None, exactly what the OSS bridge returns, so the test pins profile resolution identically in both trees.

Test

prompt_studio/prompt_studio_core_v2/tests/test_profile_resolution_fallback.py
13 passed

🤖 Generated with Claude Code

The profile-resolution tests added in #2203 patch every collaborator of
build_fetch_response_payload except get_lookup_config. In OSS that
bridge is a no-op, so the suite is green here — but cloud CI runs this
same suite against the merged tree, where the bridge resolves to the
real lookups implementation and feeds the test's MagicMock prompt into
an ORM query. Since 892be32 landed, every cloud run fails four of
these tests with:

    django.core.exceptions.ValidationError: ['"[]" is not a valid UUID.']

Patch the bridge out like every other collaborator (returning None,
exactly what the OSS bridge does), so the suite pins profile resolution
in both trees.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f093a5fb-dcd2-45c3-824d-018e4d1aee42

📥 Commits

Reviewing files that changed from the base of the PR and between 6be8b55 and 1926145.

📒 Files selected for processing (1)
  • backend/prompt_studio/prompt_studio_core_v2/tests/test_profile_resolution_fallback.py

Summary by CodeRabbit

  • Tests
    • Improved automated test reliability by isolating profile resolution scenarios from external data lookups.
    • Standardized validation test setup while preserving existing behavior.
    • No user-facing functionality or workflow changes are included in this update.

Walkthrough

The test setup now mocks get_lookup_config to prevent ORM lookups during payload-builder tests. The existing validation patch formatting was also updated without changing behavior.

Changes

Profile resolution test isolation

Layer / File(s) Summary
Mock lookup configuration in payload-builder tests
backend/prompt_studio/prompt_studio_core_v2/tests/test_profile_resolution_fallback.py
The payload-builder test setup returns None from get_lookup_config. The validate_profile_manager_owner_access patch was reformatted without behavior changes.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Suggested reviewers: hari-kuriakose

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the change, cause, implementation, and testing, but omits the required feature-breakage assessment and several template sections. Add the required breakage statement and complete the Database Migrations, Env Config, Relevant Docs, Dependencies Versions, Checklist, and related sections.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the test fix and its cloud CI purpose.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/profile-fallback-test-cloud-overlay

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

sonarqubecloud Bot commented Aug 6, 2026

Copy link
Copy Markdown

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR isolates the lookup collaborator in profile-resolution fallback tests so the same tests remain deterministic in OSS and cloud trees.

  • Patches get_lookup_config to return the OSS no-op value.
  • Reformats an existing patch call without changing behavior.

Confidence Score: 5/5

The PR appears safe to merge with no actionable defects identified.

The new patch targets the exact lookup binding invoked by the tested payload builder, returns the established OSS no-op value, and does not bypass the profile-resolution logic or assertions.

Important Files Changed

Filename Overview
backend/prompt_studio/prompt_studio_core_v2/tests/test_profile_resolution_fallback.py Correctly patches the module-level lookup binding used by the payload builder while preserving the profile-resolution behavior under test.

Reviews (1): Last reviewed commit: "[FIX] Patch the lookups bridge in the pr..." | Re-trigger Greptile

@pk-zipstack

Copy link
Copy Markdown
Contributor Author

Closing for now — will discuss with the author of #2203 and follow up. The diagnosis stands for reference: since 892be32, cloud CI (which runs this suite against the merged tree) fails 4 of these tests because the real lookups implementation receives the test's MagicMock prompt — first failing cloud-main run: https://github.com/Zipstack/unstract-cloud/actions/runs/31078494222.

@pk-zipstack pk-zipstack closed this Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
e2e-api-deployment e2e 3 0 0 0 20.5
e2e-coowners e2e 1 0 0 0 1.5
e2e-etl e2e 1 0 0 0 8.8
e2e-login e2e 2 0 0 0 1.2
e2e-prompt-studio e2e 1 0 0 0 4.5
e2e-smoke e2e 2 0 0 0 2.7
e2e-workflow e2e 1 0 0 0 16.3
integration-backend integration 205 0 0 26 42.4
integration-connectors integration 1 0 0 7 7.8
integration-workers integration 140 0 0 1 50.3
unit-backend unit 460 0 0 1 34.7
unit-connectors unit 63 0 0 0 9.3
unit-core unit 33 0 0 0 1.2
unit-platform-service unit 15 0 0 0 2.5
unit-rig unit 109 0 0 0 5.0
unit-sdk1 unit 480 0 0 0 24.9
unit-workers unit 1335 0 0 1 99.6
TOTAL 2852 0 0 36 333.3

Critical paths

⚠️ Critical paths not yet covered

  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (declared coverage: no groups declared)
✅ Covered critical paths
  • auth-login — covered by e2e-login
  • adapter-register-llm — covered by integration-backend
  • workflow-author — covered by integration-backend
  • co-owner-manage — covered by integration-backend, e2e-coowners
  • workflow-create-execute — covered by e2e-workflow
  • api-deployment-provision — covered by integration-backend
  • api-deployment-auth — covered by integration-backend
  • api-deployment-run — covered by e2e-api-deployment
  • prompt-studio-author — covered by integration-backend
  • prompt-studio-fetch-response — covered by e2e-prompt-studio
  • connector-register-test — covered by integration-backend
  • pipeline-etl-execute — covered by e2e-etl
  • usage-aggregate-read — covered by integration-backend
  • usage-token-tracking — covered by e2e-api-deployment
  • callback-result-delivery — covered by e2e-api-deployment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant