Extend resolution search labels and GeoJSON annotations#662
Conversation
…ted contextual labels and semantic GeoJSON annotations. - Updated \`resolutionSearchSchema\` to include \`mapboxImageLabel\`, \`googleImageLabel\`, and \`analysisFocus\`. - Enhanced GeoJSON features with \`featureCategory\` and \`displayLabel\`. - Updated agent system prompt to support dynamic labeling and semantic map annotations. - Wired dynamic labels through \`ResolutionCarousel\` and \`CompareSlider\` components. - Integrated detected features context into \`querySuggestor\` for improved follow-up queries. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
WalkthroughThis PR adds contextual labels and semantic feature annotations to the map-resolution workflow. The schema is extended with feature categories and location-aware labels. The ChangesMap Resolution Contextual Labels and Feature Annotations
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related issues
Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/actions.tsx`:
- Line 185: Replace the use of the unsafe any in the detectedFeatures mapping by
typing the feature parameter with the schema-derived type; e.g., import or
reference the ResolutionSearch type (from resolutionSearchSchema) and change (f:
any) to (f: ResolutionSearch["geoJson"]["features"][number]) or create a local
alias like type Feature =
NonNullable<ResolutionSearch["geoJson"]>["features"][number] and use (f:
Feature) so TypeScript infers properties like name and featureCategory safely
when computing detectedFeatures.
In `@lib/agents/resolution-search.tsx`:
- Around line 131-133: Summary: The prompt requires every geoJson feature to
"must have" featureCategory and displayLabel but the schema (featureCategory,
displayLabel) marks them optional, causing a mismatch. Fix: choose Option 1
(recommended) — update the prompt text in the resolution-search prompt generator
to state that featureCategory and displayLabel are optional and should be
provided when available (or include sensible fallbacks), and ensure the prompt
references mapboxImageLabel, googleImageLabel, and analysisFocus accurately;
alternatively, if you prefer Option 2, change the zod schema definitions for
featureCategory and displayLabel in the resolution-search schema to required
enums/strings (replace optional() with required definitions) so they match the
prompt; finally, verify any code that serializes geoJson respects the chosen
behavior (use Point vs Polygon and include name, featureCategory, displayLabel
when present).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2b28de3b-5469-41f7-9a0a-667600deb9b7
📒 Files selected for processing (6)
app/actions.tsxcomponents/compare-slider.tsxcomponents/resolution-carousel.tsxlib/agents/query-suggestor.tsxlib/agents/resolution-search.tsxlib/schema/resolution-search.ts
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-04-25T08:10:37.673Z
Learning: In the QCX repository (feat/ai-sdk-ui-migration branch and going forward), there is no token generation/counting tracking and no multimodal LLM features. Base64 image data sent via `append()` body fields in components like `header-search-button.tsx` and `resolution-carousel.tsx` is for server-side `resolution_search` processing only — it is NOT passed as multimodal message content parts to a vision model. Avoid flagging token tracking or multimodal LLM handling gaps as review concerns in this codebase.
📚 Learning: 2026-01-16T10:02:40.220Z
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-01-16T10:02:40.220Z
Learning: The router agent and resolution search agent must implement a feedback loop where tool execution results (including GeoTIFF URLs, embeddings, and satellite analysis data) are fed back to the AI model's context through enriched messages before generating the final response, ensuring the foundational model receives and can synthesize all satellite intelligence data.
Applied to files:
lib/agents/resolution-search.tsx
📚 Learning: 2026-04-25T08:10:37.673Z
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-04-25T08:10:37.673Z
Learning: In the QCX repository (feat/ai-sdk-ui-migration branch and going forward), there is no token generation/counting tracking and no multimodal LLM features. Base64 image data sent via `append()` body fields in components like `header-search-button.tsx` and `resolution-carousel.tsx` is for server-side `resolution_search` processing only — it is NOT passed as multimodal message content parts to a vision model. Avoid flagging token tracking or multimodal LLM handling gaps as review concerns in this codebase.
Applied to files:
lib/agents/resolution-search.tsxcomponents/resolution-carousel.tsxapp/actions.tsxlib/agents/query-suggestor.tsx
📚 Learning: 2026-02-08T16:50:33.791Z
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-02-08T16:50:33.791Z
Learning: The PR branch implements RAG (Retrieval-Augmented Generation) via `lib/actions/rag.ts` which uses Supabase RPC functions `generate_embedding()` and `hybrid_search()` to retrieve relevant context from past conversations. However, there's a bug on line 433 of `app/actions.tsx` where `currentSystemPrompt` is passed to the researcher instead of `augmentedSystemPrompt`, preventing the retrieved context from being used.
Applied to files:
lib/agents/query-suggestor.tsx
🔇 Additional comments (8)
lib/schema/resolution-search.ts (1)
22-37: LGTM!lib/agents/query-suggestor.tsx (1)
17-27: LGTM!Also applies to: 29-33, 61-61
components/compare-slider.tsx (1)
10-11: LGTM!Also applies to: 14-14, 85-90
components/resolution-carousel.tsx (1)
25-27: LGTM!Also applies to: 30-37, 88-89, 93-93, 115-117, 135-136, 152-154
app/actions.tsx (4)
144-146: LGTM!Also applies to: 185-187
214-217: LGTM!
849-851: LGTM!Also applies to: 861-863
246-258: Update streaming UI after analysis to show labels (or explain why reload is required).In
app/actions.tsx, the streaminguiStream.updaterendersResolutionCarouselwithmapboxImageLabelandgoogleImageLabelset toundefined. If labels are only written to AI state after analysis anduiStreamis not updated again, the user won’t see labels until a full reload viagetUIStateFromAIState.Can the codebase update the streaming UI with post-analysis results (e.g., via a later
uiStream.update/uiStream.append) in the same request, or is the “labels appear only after reload” behavior intentional UX? If there’s no later UI update, consider updating beforeuiStream.done()so the analysis results become visible immediately.
| return m | ||
| }); | ||
| const relatedQueries = await querySuggestor(uiStream, sanitizedMessages); | ||
| const detectedFeatures = analysisResult.geoJson?.features?.map((f: any) => `${f.name} (${f.featureCategory || "other"})`).join(", "); |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | 💤 Low value
Consider stronger typing for feature mapping.
The code uses (f: any) which reduces type safety. Since analysisResult is of type ResolutionSearch (inferred from resolutionSearchSchema), the features array has a known structure.
♻️ Optional improvement for type safety
-const detectedFeatures = analysisResult.geoJson?.features?.map((f: any) => `${f.name} (${f.featureCategory || "other"})`).join(", ");
+const detectedFeatures = analysisResult.geoJson?.features?.map((f) => `${f.name} (${f.featureCategory || "other"})`).join(", ");TypeScript will infer the correct type from the schema, providing better autocomplete and type checking.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const detectedFeatures = analysisResult.geoJson?.features?.map((f: any) => `${f.name} (${f.featureCategory || "other"})`).join(", "); | |
| const detectedFeatures = analysisResult.geoJson?.features?.map((f) => `${f.name} (${f.featureCategory || "other"})`).join(", "); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/actions.tsx` at line 185, Replace the use of the unsafe any in the
detectedFeatures mapping by typing the feature parameter with the schema-derived
type; e.g., import or reference the ResolutionSearch type (from
resolutionSearchSchema) and change (f: any) to (f:
ResolutionSearch["geoJson"]["features"][number]) or create a local alias like
type Feature = NonNullable<ResolutionSearch["geoJson"]>["features"][number] and
use (f: Feature) so TypeScript infers properties like name and featureCategory
safely when computing detectedFeatures.
| 8. **Contextual Labeling:** Produce location-specific and feature-aware labels for the images (\`mapboxImageLabel\`, \`googleImageLabel\`, \`analysisFocus\`). Reference user-drawn features in these labels when they are present. Labels should be concise, uppercase-friendly, and descriptive of the analysis focus. Examples: "Analysis of drawn area: [feature type]", "[Location name] satellite view". | ||
| 9. **Semantic Annotations:** When creating \`geoJson\` features, use Point features for POIs and landmarks, and Polygon features for land areas or regions of interest. Each feature must have a \`name\`, \`featureCategory\`, and \`displayLabel\` matching findings in your summary. Example: Create a Point for identified bridges with name 'Main Street Bridge', category 'infrastructure'. | ||
|
|
There was a problem hiding this comment.
Align prompt instruction with schema optionality.
The prompt states that each feature "must have" featureCategory and displayLabel, but the schema defines these fields as optional (lines 23-26 in lib/schema/resolution-search.ts). This mismatch could lead to inconsistent model behavior or confusion about validation requirements.
🔧 Recommended alignment options
Option 1 (recommended): Update the prompt to reflect optionality:
-9. **Semantic Annotations:** When creating \`geoJson\` features, use Point features for POIs and landmarks, and Polygon features for land areas or regions of interest. Each feature must have a \`name\`, \`featureCategory\`, and \`displayLabel\` matching findings in your summary. Example: Create a Point for identified bridges with name 'Main Street Bridge', category 'infrastructure'.
+9. **Semantic Annotations:** When creating \`geoJson\` features, use Point features for POIs and landmarks, and Polygon features for land areas or regions of interest. Each feature must have a \`name\` and should include \`featureCategory\` and \`displayLabel\` when applicable to match findings in your summary. Example: Create a Point for identified bridges with name 'Main Street Bridge', category 'infrastructure', and displayLabel 'Main St. Bridge'.Option 2: Update the schema to make fields required:
// In lib/schema/resolution-search.ts
featureCategory: z.enum(['poi', 'land_feature', 'infrastructure', 'drawn_area', 'other'])
.describe('The category of the feature...'),
displayLabel: z.string().describe('A short label for map display or tooltips')(Note: Option 1 is safer since downstream code already handles these as optional with fallbacks.)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/agents/resolution-search.tsx` around lines 131 - 133, Summary: The prompt
requires every geoJson feature to "must have" featureCategory and displayLabel
but the schema (featureCategory, displayLabel) marks them optional, causing a
mismatch. Fix: choose Option 1 (recommended) — update the prompt text in the
resolution-search prompt generator to state that featureCategory and
displayLabel are optional and should be provided when available (or include
sensible fallbacks), and ensure the prompt references mapboxImageLabel,
googleImageLabel, and analysisFocus accurately; alternatively, if you prefer
Option 2, change the zod schema definitions for featureCategory and displayLabel
in the resolution-search schema to required enums/strings (replace optional()
with required definitions) so they match the prompt; finally, verify any code
that serializes geoJson respects the chosen behavior (use Point vs Polygon and
include name, featureCategory, displayLabel when present).
2 participants
Extend the resolution search schema and agent system prompt to support AI-generated contextual labels and semantic map annotations. Wire these labels through the UI and use detected features to improve follow-up query suggestions.
Changes:
PR created automatically by Jules for task 15821560493438138230 started by @ngoiyaeric
Summary by CodeRabbit