Skip to content

UNOMI-879 Unified CRUD Karaf shell developer commands (includes UNOMI-920 YAML toString() debugging)#755

Merged
sergehuber merged 36 commits into
masterfrom
UNOMI-879-unified-crud-shell
Jun 2, 2026
Merged

UNOMI-879 Unified CRUD Karaf shell developer commands (includes UNOMI-920 YAML toString() debugging)#755
sergehuber merged 36 commits into
masterfrom
UNOMI-879-unified-crud-shell

Conversation

@sergehuber
Copy link
Copy Markdown
Contributor

@sergehuber sergehuber commented May 6, 2026

Stacked PR (merge order)

Role Branch
Base (merge into) UNOMI-920-yaml-tostring
Head (this PR) UNOMI-879-unified-crud-shell

This PR is stacked: merge into UNOMI-920-yaml-tostring first (after #754 is merged to master, or equivalent).

This branch is stacked: it builds on the UNOMI-920 commit, then adds the UNOMI-879 shell work.
For more information (each PR targets the branch below until the bottom merges): https://github.github.com/gh-stack/introduction/overview/


JIRA


UNOMI-879 — Unified CRUD shell (shell-dev-commands)

The previous Karaf shell commands grew inconsistent across object types (list / view style varied), duplicated a lot of logic, used different arguments and behaviours for similar operations, and scattered help and docs, which raised maintenance cost and made the shell harder for new operators and contributors. The unified unomi:crud style (operations × types, shared table/CSV output, JSON create/update, and coherent completion/help) matches UNOMI-879: one predictable pattern, shared plumbing for listing and errors, and less duplicated command code so fixes and features apply everywhere at once.

Replaces the older fragmented list/view-style commands with a consistent CRUD-oriented layout under org.apache.unomi.shell.dev: shared routing (UnomiCrudCommand), CrudCommand / BaseCrudCommand, table/CSV listing support, completers, and type-specific command classes (rules, segments, profiles, events, sessions, definitions, etc.). Deployment / tail / watch helpers are moved into the same dev.commands package alongside the new commands.

API support (aligned with unomi-3-dev behaviour):

  • EventService#deleteEvent(String) / EventServiceImpl — remove event by id via persistence.
  • ProfileService#deleteSession(String) / ProfileServiceImpl — remove session by id via persistence.

Shell wiring depends on these for EventCrudCommand / SessionCrudCommand delete operations.

Build: bom / shell-dev-commands pom updates as needed for the module.

sergehuber added 2 commits May 5, 2026 17:17
- Add YamlUtils (SnakeYaml) with YamlConvertible, YamlMapBuilder, circular
  reference detection (identity-based visited sets), and recursion depth limits.
- Implement YAML-backed toString()/toYaml() on core API types extending Item /
  MetadataItem (Condition, ConditionType, Action, ActionType, Rule, Segment,
  Goal, Scoring, ScoringElement, Parameter, Metadata, etc.).
- Add YamlUtilsTest coverage for builder, toYamlValue, and representative rules.

Build and integration alignment:
- unomi-api: snakeyaml + test dependencies; manage mockito-core version in BOM.
- RESTParameter: use Object for defaultValue to match Parameter#getDefaultValue().
- RulesServiceImpl: avoid NPE when tracked parameter defaultValue is null before split.
- itests: override awaitility to 3.1.6 for OSGi (Hamcrest 1.3 bundle); Karaf
  itests common logback exclusions; hamcrest bundle scope test.
- Replace legacy per-type list/view commands with unified unomi:crud routing,
  shared bases (BaseCrudCommand, CrudCommand), completers, and type-specific
  CRUD implementations under org.apache.unomi.shell.dev.
- Relocate deployment/tail/watch helpers into dev.commands package.
- EventService/ProfileService: add deleteEvent/deleteSession; implementations
  delegate to PersistenceService.remove (aligned with unomi-3-dev).
- BOM: dependency adjustments needed by shell-dev-commands build.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes and unifies the Karaf “developer” shell commands by introducing a single unomi:crud command that routes CRUD/list/help operations to type-specific handlers, while also adding backend API support to delete persisted Events and Sessions (needed by the new delete operations).

Changes:

  • Added a CrudCommand SPI plus a shared BaseCrudCommand implementation for consistent list/table/CSV output across object types.
  • Introduced the unomi:crud command with operation/type/id completers and a set of new type-specific CRUD handlers (rules, segments, profiles, events, sessions, goals, etc.), while removing legacy fragmented commands.
  • Extended EventService / ProfileService APIs and implementations with deleteEvent / deleteSession, and updated Maven/OSGi metadata for the shell module.

Reviewed changes

Copilot reviewed 62 out of 62 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/services/CrudCommand.java New SPI for CRUD handlers used by unomi:crud.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/services/BaseCrudCommand.java Shared list/query/table/CSV scaffolding for CRUD handlers.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/TypeCompleter.java Completes available CRUD object types from OSGi services.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/OperationCompleter.java Completes supported operations for unomi:crud.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/IdCompleter.java Completes IDs for read/update/delete contexts.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/BaseCompleter.java Base completer for item-id completions (existing pattern reused).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/UndeployDefinition.java Moved/rewired undeploy command into dev.commands.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/topics/TopicCrudCommand.java CRUD handler for topics.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/TailCommandUtils.java Shared helpers for tail/watch output formatting.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/TailCommandSupport.java Repackaged tail support into dev.commands and aligned console output.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/sessions/SessionCrudCommand.java CRUD handler for sessions (uses new delete API).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/segments/SegmentCrudCommand.java CRUD handler for segments.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/scoring/ScoringCrudCommand.java CRUD handler for scoring definitions.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/scopes/ScopeCrudCommand.java CRUD handler for scopes.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleWatch.java Repackaged rule watch command into dev.commands.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleTail.java Repackaged rule tail command into dev.commands.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/rules/RuleStatisticsCrudCommand.java CRUD handler for rule statistics.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/rules/RuleCrudCommand.java CRUD handler for rules (includes statistics columns).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleResetStats.java Updated reset-stats command to use shared console base.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RemoveCommandSupport.java New shared “remove with confirmation” base command.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/properties/PropertyTypeCrudCommand.java CRUD handler for property types.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/profiles/ProfileCrudCommand.java CRUD handler for profiles (plus ID completion).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/profiles/ProfileAliasCrudCommand.java CRUD handler for profile aliases.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/personas/PersonaCrudCommand.java CRUD handler for personas.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/ListCommandSupport.java Repackaged list support and fixed output to session console.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/goals/GoalCrudCommand.java CRUD handler for goals.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/EventTail.java Repackaged event tail command and reused shared formatting.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/events/EventCrudCommand.java CRUD handler for events (uses new delete API).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/DeploymentCommandSupport.java Refactored deployment support into dev.commands with shared console base.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/DeployDefinition.java New deploy-definition command under dev.commands.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/consents/ConsentCrudCommand.java CRUD handler for consents (profile-embedded).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/conditions/ConditionTypeCrudCommand.java CRUD handler for condition types.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/CommandUtils.java Shared command utilities (date formatting).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/campaigns/CampaignEventCrudCommand.java CRUD handler for campaign events.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/campaigns/CampaignCrudCommand.java CRUD handler for campaigns.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseSimpleCommand.java Base class for commands needing consistent session console output.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseListCommand.java New generic list base (shared query/table pattern).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseCommand.java New generic command base utilities (table + confirm).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/actions/ActionTypeCrudCommand.java CRUD handler for action types.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/actions/UnomiCrudCommand.java New unified unomi:crud router command.
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/UndeployDefinition.java Removed legacy command (replaced by dev.commands).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SessionList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentView.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentRemove.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/RuleList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ProfileView.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ProfileList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventView.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventSearch.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/DeployDefinition.java Removed legacy command (replaced by dev.commands).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ConditionView.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ConditionList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ActionView.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ActionList.java Removed legacy command (replaced by CRUD).
tools/shell-dev-commands/pom.xml Adds commons-csv and adjusts OSGi bundle/plugin metadata.
services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java Implements new ProfileService#deleteSession().
services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java Implements new EventService#deleteEvent().
bom/pom.xml Adds an additional managed dependency entry for httpclient-osgi with type bundle.
api/src/main/java/org/apache/unomi/api/services/ProfileService.java Adds ProfileService#deleteSession API.
api/src/main/java/org/apache/unomi/api/services/EventService.java Adds EventService#deleteEvent API.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread bom/pom.xml Outdated
sergehuber and others added 16 commits May 15, 2026 08:10
Run mikepenz/action-junit-report after every integration matrix leg
(if: always), with update_check and per-matrix check_name, so a green
re-run replaces the stale red JUnit check. Use fail_on_failure=false and
require_tests=false; continue-on-error so missing XML cannot fail the job.
Use identity-based visited sets for circular-reference detection, harden
Condition deepCopy and parameter handling, fix Parameter max-depth YAML,
wire slf4j.version in unomi-api, and expand unit test coverage.
Fix condition undeploy, rule list performance, null-safety and pagination
in CRUD commands, thread-safe consent date formatting, read-only rulestats
behavior, segment ID completion, and BOM httpclient dependency management.
Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary
download + forked JVM) with io.fabric8:docker-maven-plugin so the
elasticsearch profile of itests runs ES in a Docker container, mirroring
how the opensearch profile already runs OpenSearch.

itests/pom.xml (elasticsearch profile)
* Add an <elasticsearch.port>9400</elasticsearch.port> property and pass
  it through the failsafe systemPropertyVariables so tests resolve the
  HTTP port from a single source.
* Replace the elasticsearch-maven-plugin block with a docker-maven-plugin
  block that starts/stops a docker.elastic.co/elasticsearch/elasticsearch
  container, binds target/snapshots_repository to /tmp/snapshots_repository,
  and waits on the HTTP port before the integration-test phase.
* Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step:
  the ES container runs as UID 1000, so on Linux CI the bind-mounted
  snapshot repo otherwise hits access_denied during repository verify.

pom.xml (root)
* Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>
  and add the pluginManagement entry so the elasticsearch profile (and
  any future user of the plugin) inherits a single version.

This is phase A0 of the PR #757 stack split (see
docs/PR-757-stack-extraction-tracker.md). It is intentionally small and
self-contained: no test/code changes, only the test infrastructure switch.
…icsearch in integration tests

Implements the core configuration switch from UNOMI-921:
https://issues.apache.org/jira/browse/UNOMI-921

Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary
download + forked JVM) with io.fabric8:docker-maven-plugin in the
elasticsearch profile of itests, mirroring how the opensearch profile
already runs OpenSearch in a Docker container.

itests/pom.xml (elasticsearch profile)
* Add an <elasticsearch.port>9400</elasticsearch.port> property and pass
  it through the failsafe systemPropertyVariables so tests resolve the
  HTTP port from a single source (unchanged from the previous 9400).
* Replace the elasticsearch-maven-plugin block with a docker-maven-plugin
  block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version},
  binds target/snapshots_repository to /tmp/snapshots_repository, and
  waits on the HTTP port before the integration-test phase. Heap aligned
  to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the
  ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and
  xpack.security disabled.
* Container lifecycle matches OpenSearch exactly: pre-integration-test
  runs stop+remove then start (with showLogs); post-integration-test
  runs stop only -- container is kept around for inspection.
* Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step:
  the ES container runs as UID 1000, so on Linux CI the bind-mounted
  snapshot repo otherwise hits access_denied during repository verify.

pom.xml (root)
* Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>
  and add the pluginManagement entry so the elasticsearch profile (and
  any future user of the plugin) inherits a single version.

Scope kept minimal for the PR #757 stack split: only the test
infrastructure switch lives here. The follow-up UNOMI-921 acceptance
items below ship in the platform PR (P) once it lands:

* Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in
  checkSearchEngine() (no longer needed with Docker).
* Migrate16xToCurrentVersionIT: replace hardcoded
  ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort().
* Drop the comments referring to the elasticsearch-maven-plugin
  template-override workaround.

See docs/PR-757-stack-extraction-tracker.md for the full split plan and
how this PR fits in the stack.
Non-interactive prompts when CI, GITHUB_ACTIONS, or BUILD_NON_INTERACTIVE is set.
Add --ci (no Karaf, no Maven build cache) and MAVEN_EXTRA_OPTS for matrix ports.
sergehuber added 10 commits May 23, 2026 10:32
Replace fixed sleeps in GraphQLListIT and poll after patch refresh in PatchIT.
Poll profile properties after events in PropertiesUpdateActionIT.
Align CI with local developer workflow; pass matrix ports via MAVEN_EXTRA_OPTS.
…icsearch in integration tests

Implements the core configuration switch from UNOMI-921:
https://issues.apache.org/jira/browse/UNOMI-921

Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary
download + forked JVM) with io.fabric8:docker-maven-plugin in the
elasticsearch profile of itests, mirroring how the opensearch profile
already runs OpenSearch in a Docker container.

itests/pom.xml (elasticsearch profile)
* Add an <elasticsearch.port>9400</elasticsearch.port> property and pass
  it through the failsafe systemPropertyVariables so tests resolve the
  HTTP port from a single source (unchanged from the previous 9400).
* Replace the elasticsearch-maven-plugin block with a docker-maven-plugin
  block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version},
  binds target/snapshots_repository to /tmp/snapshots_repository, and
  waits on the HTTP port before the integration-test phase. Heap aligned
  to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the
  ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and
  xpack.security disabled.
* Container lifecycle matches OpenSearch exactly: pre-integration-test
  runs stop+remove then start (with showLogs); post-integration-test
  runs stop only -- container is kept around for inspection.
* Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step:
  the ES container runs as UID 1000, so on Linux CI the bind-mounted
  snapshot repo otherwise hits access_denied during repository verify.

pom.xml (root)
* Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>
  and add the pluginManagement entry so the elasticsearch profile (and
  any future user of the plugin) inherits a single version.

Scope kept minimal for the PR #757 stack split: only the test
infrastructure switch lives here. The follow-up UNOMI-921 acceptance
items below ship in the platform PR (P) once it lands:

* Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in
  checkSearchEngine() (no longer needed with Docker).
* Migrate16xToCurrentVersionIT: replace hardcoded
  ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort().
* Drop the comments referring to the elasticsearch-maven-plugin
  template-override workaround.

See docs/PR-757-stack-extraction-tracker.md for the full split plan and
how this PR fits in the stack.
# Conflicts:
#	build.sh
#	itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
#	itests/src/test/java/org/apache/unomi/itests/graphql/GraphQLListIT.java
@sergehuber sergehuber changed the base branch from UNOMI-920-yaml-tostring to master May 30, 2026 10:03
# Conflicts:
#	api/src/main/java/org/apache/unomi/api/conditions/Condition.java
#	api/src/main/java/org/apache/unomi/api/utils/YamlUtils.java
#	api/src/test/java/org/apache/unomi/api/conditions/ConditionTest.java
#	build.sh
#	itests/pom.xml
#	itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
- Remove duplicate toYaml/deepCopy block in Condition.java (compile error)
- Fix OSGi service reference leak in UnomiCrudCommand.findAndExecuteCommand
- Fix CSVPrinter closing Karaf session stream (flush instead of close)
- Replace new Integer(String) with Integer.parseInt() (removed in Java 17)
- Fix paginated read() in CampaignEventCrudCommand and ProfileAliasCrudCommand
- Fix silent swallow in ConsentCrudCommand.update() and no-op in Campaign/GoalCrudCommand.update()
- Fix ProfileCrudCommand.create() ignoring user-supplied itemId
- Fix UndeployDefinition PATCH case returning false success
- Add null guards in TailCommandUtils for null timestamps
- Fix GraphQLListIT compilation error and assertion message
- Minor cleanup: unused imports, Javadoc, vestigial throws declaration
@sergehuber sergehuber merged commit 956440c into master Jun 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants