feat: new manifest format by atilafassina · Pull Request #261 · databricks/appkit

atilafassina · 2026-04-08T16:31:10Z

This branch evolves the plugin manifest system from v1.0 to v2.0, adding three major capabilities to support smarter scaffolding by the databricks apps init CLI.

Schema Extensions (commit `4c42edd`)

discoveryDescriptor — new schema definition describing how the CLI auto-discovers resource field values via Databricks CLI commands (e.g.
listing warehouses). Supports cliCommand (with placeholder), selectField/displayField for interactive selection, dependsOn for
inter-field ordering, and a shortcut for single-value fast paths.
postScaffoldStep — ordered instructions shown to the user after project init (e.g. "run migrations", "configure secrets").
origin field on resource entries — computed value indicating where a field's value comes from ("platform", "static", "cli", or "user").

Origin Computation & Scaffolding Descriptor (commit `7fe3d74`)

computeOrigin() derives origin from field properties:
- local only → platform
- value present → static
- resolve present → cli
- user input → user.
enrichFieldsWithOrigin() injects computed origin into every resource field at sync time.
Adds a scaffolding descriptor to the template manifest with the databricks apps init command signature, flags, and guardrail rules
(never/must).
Bumps manifest version from "1.0" → "2.0".

Semantic Validation (commit `b905167`)

Four new cross-field validators added to appkit plugin validate:

dependsOn cycle detection — DFS-based check for circular dependencies between discovery fields.
dependsOn dangling reference — errors if a field references a non-existent sibling.
Discovery <PROFILE> placeholder — ensures every cliCommand includes the placeholder.
Discovery/origin coherence — warns if a field has discovery but its computed origin isn't "user" (meaning discovery would never be used).
postScaffold structural validation — verifies array structure and non-empty instructions.

Core Plugin Annotations (commit `d8cbbf4`)

Annotates all four built-in plugin manifests (analytics, files, genie, lakebase) with:

discovery descriptors on resource fields (e.g. warehouse listing, catalog/schema browsing)
postScaffold steps (e.g. "Create a SQL warehouse", "Enable Lakebase")

Schema Fix (commit `eb65776`)

Inlines resourceFieldEntry and resourceRequirement definitions in the template schema so origin and discovery are properly represented without $ref resolution issues.

Test Results

Run 1

Metric	Delta	Value
Generated	--	100% (5/5)
Eval Score	↑ 0.03	1.00
Gen Turns	↓ 4.40	41.2
Build Pass	--	5 / 5
Tests Pass	↑ 1	10 (U5 S5)
Edit Gen %	--	100% (4/4)
Edit Score	--	100% (4/4)
Edit Turns	↑ 2	22.5
Edit Tests	--	4 / 4

…tScaffold, and scaffolding Xavier loop: iteration 1 — Phase 1 (Schema Definitions & Type Generation) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

…te manifest emission Xavier loop: iteration 2 — Phase 2 (Origin Computation & Sync Enrichment) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

…, and postScaffold Xavier loop: iteration 3 — Phase 3 (Semantic Validation) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

…ostScaffold steps Xavier loop: iteration 4 — Phase 4 (Core Plugin Manifest Annotations) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

…t for origin support JSON Schema Draft-07 additionalProperties:false blocks allOf composition. Inlined both defs in template schema so origin validates correctly. Xavier loop: iteration 5 — Phase 5 (Integration & Backpressure) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

Copilot

Pull request overview

This PR upgrades the AppKit plugin manifest ecosystem to a v2.0 template manifest format to support smarter databricks apps init scaffolding, including field discovery metadata, post-scaffold user instructions, computed field origins, and semantic (cross-field) validation during plugin validate.

Changes:

Bump template plugin manifest version to 2.0 and add a top-level scaffolding descriptor.
Extend plugin/template schemas with discovery (CLI command-based value discovery) and postScaffold steps; generate/propagate computed origin into template manifests during sync.
Add semantic validation (dependsOn cycles/dangling refs, <PROFILE> placeholder, discovery/origin coherence, postScaffold structure) and associated tests/docs updates.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
template/appkit.plugins.json	Updates template manifest to v2.0 and annotates built-in plugins with discovery/origin/postScaffold plus scaffolding descriptor.
packages/shared/src/schemas/template-plugins.schema.json	Expands template manifest schema to support v2.0 and scaffolding descriptor; inlines field/requirement defs for origin.
packages/shared/src/schemas/plugin-manifest.schema.json	Adds `discovery` to resource fields and `postScaffold` to plugin manifests.
packages/shared/src/schemas/plugin-manifest.generated.ts	Updates generated TS types to include discovery/postScaffold types.
packages/shared/src/plugin.ts	Re-exports new generated types (DiscoveryDescriptor, PostScaffoldStep).
packages/shared/src/cli/commands/plugin/validate/validate.ts	Runs semantic validation and formats semantic errors/warnings in CLI output.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.ts	Implements semantic validation rules and issue formatting.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.test.ts	Adds test coverage for new semantic validation behavior.
packages/shared/src/cli/commands/plugin/sync/sync.ts	Computes/injects `origin`, bumps template manifest to v2.0, and adds scaffolding descriptor on write.
packages/shared/src/cli/commands/plugin/sync/sync.test.ts	Adds unit tests for origin computation.
packages/shared/src/cli/commands/plugin/manifest-types.ts	Adds scaffolding descriptor types and exports new manifest-related types.
packages/appkit/src/plugins/lakebase/manifest.json	Adds discovery descriptors and postScaffold steps to the lakebase built-in plugin manifest.
packages/appkit/src/plugins/genie/manifest.json	Adds schema reference, discovery descriptor, and postScaffold steps to genie plugin manifest.
packages/appkit/src/plugins/files/manifest.json	Adds discovery descriptor and postScaffold steps to files plugin manifest.
packages/appkit/src/plugins/analytics/manifest.json	Adds discovery descriptor and postScaffold steps to analytics plugin manifest.
docs/static/schemas/template-plugins.schema.json	Publishes updated template plugins schema for docs site.
docs/static/schemas/plugin-manifest.schema.json	Publishes updated plugin manifest schema for docs site.
docs/static/appkit-ui/styles.gen.css	Updates generated UI styles (tailwind output) used by docs UI.
docs/docs/api/appkit/Interface.ResourceFieldEntry.md	Documents the new `discovery` field on ResourceFieldEntry.
docs/docs/api/appkit/Interface.PluginManifest.md	Documents the new `postScaffold` field on PluginManifest.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…r-template xavier loop: iteration 1 — Phase 1 complete

keugenek · 2026-04-20T15:17:38Z

@atilafassina this one looks ok and evolution, however little heavy code-wise (let me post comments from isaac about these) and mixing concepts little bit. What do you think of this:

Alternatives worth weighing

Zod as single source of truth (my default for a TS-first ecosystem):

Types, runtime validation, and semantic checks in one file.
zod-to-json-schema emits the $schema JSON for VSCode intellisense.
Cycle/dangling-ref checks become .refine() calls in the same module as the
schema.
Deletes plugin-manifest.generated.ts and the Ajv layer.

TS-authored manifests compiled to JSON:

manifest.ts → defineManifest({...}) with branded types for field names
(compile-time dependsOn checking, no DFS needed).
sync lowers to .json for non-TS consumers.
Strongest author ergonomics; agent still reads JSON.

Split concerns into two files:

manifest.json — strict contract (resources, fields, env), stable schema.
scaffold.yaml or .md — agent hints (discovery, prompts, postScaffold), free
to evolve.
Today they're fused, which is why the schema keeps growing.

keugenek

Overall direction looks right — v2.0 versioning via if/then is correct and co-located manifest.json per plugin is the right locality. A few shape-level concerns worth addressing before or soon after this lands. Happy to chat on any of them.

keugenek · 2026-04-20T15:26:34Z

        }
      }
+    },
+    "discoveryDescriptor": {


cliCommand is a string template with a <PROFILE> placeholder plus a jq path in selectField. The only validation is "contains <PROFILE>" — nothing checks the CLI output shape, so paginated or wrapped responses ({warehouses:[...]}) will fail at runtime. Consider {resourceKind, select, display} and letting the CLI own the command.

keugenek · 2026-04-20T15:26:34Z

+ * Injects computed `origin` onto every resource field in all plugins.
+ * Mutates the plugins object in place for efficiency.
+ */
+function enrichFieldsWithOrigin(


origin is derived from other fields then serialized back, which is why validateDiscoveryOrigin has to warn about inconsistent states. Computing at read time would remove both the enrichment step and that validator.

keugenek · 2026-04-20T15:26:34Z

+      },
+      "additionalProperties": false
+    },
+    "scaffoldingDescriptor": {


rules.never[] / rules.must[] are LLM prompts inside the schema. No validator can check what actually matters (agent behavior), and the schema file becomes part prompt. Worth splitting into a sibling scaffold.prompts.md.

keugenek · 2026-04-20T15:26:34Z

@@ -98,10 +120,330 @@
      "$ref": "plugin-manifest.schema.json#/$defs/resourceType"
    },
    "resourceFieldEntry": {


Per commit eb65776, resourceFieldEntry / resourceRequirement were inlined instead of resolving $ref. The plugin and template schemas will drift. Better to fix the ref resolver (e.g. bundle via @apidevtools/json-schema-ref-parser at publish).

keugenek · 2026-04-20T15:26:35Z

@@ -0,0 +1 @@
+{"sessionId":"f1aff2d6-c2fa-41a1-9596-0670b1e35bda","pid":6435,"acquiredAt":1775737052293}


Accidental commit — add to .gitignore.

atilafassina added 5 commits April 8, 2026 17:49

feat: extend plugin and template manifest schemas with discovery, pos…

4c42edd

…tScaffold, and scaffolding Xavier loop: iteration 1 — Phase 1 (Schema Definitions & Type Generation) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

feat: add origin computation, scaffolding descriptor, and v2.0 templa…

7fe3d74

…te manifest emission Xavier loop: iteration 2 — Phase 2 (Origin Computation & Sync Enrichment) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

feat: add semantic validation for dependsOn cycles, discovery profile…

b905167

…, and postScaffold Xavier loop: iteration 3 — Phase 3 (Semantic Validation) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

feat: annotate core plugin manifests with discovery descriptors and p…

d8cbbf4

…ostScaffold steps Xavier loop: iteration 4 — Phase 4 (Core Plugin Manifest Annotations) Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>

atilafassina requested review from MarioCadenas, arsenyinfo, calvarjorge, keugenek and pkosiec April 9, 2026 08:13

atilafassina marked this pull request as ready for review April 9, 2026 08:16

Copilot AI review requested due to automatic review settings April 9, 2026 08:16

Copilot started reviewing on behalf of atilafassina April 9, 2026 08:17 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

Comment thread packages/shared/src/cli/commands/plugin/validate/validate-manifest.ts

Comment thread packages/shared/src/schemas/plugin-manifest.generated.ts Outdated

Comment thread template/appkit.plugins.json Outdated

Comment thread packages/appkit/src/plugins/lakebase/manifest.json Outdated

This was referenced Apr 9, 2026

[PoC] chore: poc manifest 2 #186

Closed

[PoC] chore: poc manifest 4 #187

Closed

[PoC] chore: poc manifest 3 #188

Closed

[PoC] chore: poc manifest 1 #189

Closed

atilafassina added 2 commits April 9, 2026 14:50

chore: address code reviews

d82bd3e

feat: dynamic tarball discovery and .packages/ embedding in prepare-p…

f27eed4

…r-template xavier loop: iteration 1 — Phase 1 complete

atilafassina changed the base branch from main to pkosiec/template-artifact April 10, 2026 10:21

MarioCadenas added the do not merge label Apr 10, 2026

Base automatically changed from pkosiec/template-artifact to main April 14, 2026 07:52

keugenek reviewed Apr 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: new manifest format#261

feat: new manifest format#261
atilafassina wants to merge 7 commits intomainfrom
manifest-ax

atilafassina commented Apr 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keugenek commented Apr 20, 2026

Uh oh!

keugenek left a comment

Uh oh!

keugenek Apr 20, 2026

Uh oh!

keugenek Apr 20, 2026

Uh oh!

keugenek Apr 20, 2026

Uh oh!

keugenek Apr 20, 2026

Uh oh!

keugenek Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -0,0 +1 @@
		{"sessionId":"f1aff2d6-c2fa-41a1-9596-0670b1e35bda","pid":6435,"acquiredAt":1775737052293} No newline at end of file

Conversation

atilafassina commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Schema Extensions (commit 4c42edd)

Origin Computation & Scaffolding Descriptor (commit 7fe3d74)

Semantic Validation (commit b905167)

Core Plugin Annotations (commit d8cbbf4)

Schema Fix (commit eb65776)

Test Results

Run 1

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keugenek commented Apr 20, 2026

Uh oh!

keugenek left a comment

Choose a reason for hiding this comment

Uh oh!

keugenek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

keugenek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

keugenek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

keugenek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

keugenek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

atilafassina commented Apr 8, 2026 •

edited

Loading

Schema Extensions (commit `4c42edd`)

Origin Computation & Scaffolding Descriptor (commit `7fe3d74`)

Semantic Validation (commit `b905167`)

Core Plugin Annotations (commit `d8cbbf4`)

Schema Fix (commit `eb65776`)