UNOMI-879 Unified CRUD Karaf shell developer commands (includes UNOMI-920 YAML toString() debugging)#755
Merged
Merged
Conversation
- Add YamlUtils (SnakeYaml) with YamlConvertible, YamlMapBuilder, circular reference detection (identity-based visited sets), and recursion depth limits. - Implement YAML-backed toString()/toYaml() on core API types extending Item / MetadataItem (Condition, ConditionType, Action, ActionType, Rule, Segment, Goal, Scoring, ScoringElement, Parameter, Metadata, etc.). - Add YamlUtilsTest coverage for builder, toYamlValue, and representative rules. Build and integration alignment: - unomi-api: snakeyaml + test dependencies; manage mockito-core version in BOM. - RESTParameter: use Object for defaultValue to match Parameter#getDefaultValue(). - RulesServiceImpl: avoid NPE when tracked parameter defaultValue is null before split. - itests: override awaitility to 3.1.6 for OSGi (Hamcrest 1.3 bundle); Karaf itests common logback exclusions; hamcrest bundle scope test.
- Replace legacy per-type list/view commands with unified unomi:crud routing, shared bases (BaseCrudCommand, CrudCommand), completers, and type-specific CRUD implementations under org.apache.unomi.shell.dev. - Relocate deployment/tail/watch helpers into dev.commands package. - EventService/ProfileService: add deleteEvent/deleteSession; implementations delegate to PersistenceService.remove (aligned with unomi-3-dev). - BOM: dependency adjustments needed by shell-dev-commands build.
2 tasks
There was a problem hiding this comment.
Pull request overview
This PR modernizes and unifies the Karaf “developer” shell commands by introducing a single unomi:crud command that routes CRUD/list/help operations to type-specific handlers, while also adding backend API support to delete persisted Events and Sessions (needed by the new delete operations).
Changes:
- Added a
CrudCommandSPI plus a sharedBaseCrudCommandimplementation for consistent list/table/CSV output across object types. - Introduced the
unomi:crudcommand with operation/type/id completers and a set of new type-specific CRUD handlers (rules, segments, profiles, events, sessions, goals, etc.), while removing legacy fragmented commands. - Extended
EventService/ProfileServiceAPIs and implementations withdeleteEvent/deleteSession, and updated Maven/OSGi metadata for the shell module.
Reviewed changes
Copilot reviewed 62 out of 62 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/services/CrudCommand.java | New SPI for CRUD handlers used by unomi:crud. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/services/BaseCrudCommand.java | Shared list/query/table/CSV scaffolding for CRUD handlers. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/TypeCompleter.java | Completes available CRUD object types from OSGi services. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/OperationCompleter.java | Completes supported operations for unomi:crud. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/IdCompleter.java | Completes IDs for read/update/delete contexts. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/completers/BaseCompleter.java | Base completer for item-id completions (existing pattern reused). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/UndeployDefinition.java | Moved/rewired undeploy command into dev.commands. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/topics/TopicCrudCommand.java | CRUD handler for topics. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/TailCommandUtils.java | Shared helpers for tail/watch output formatting. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/TailCommandSupport.java | Repackaged tail support into dev.commands and aligned console output. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/sessions/SessionCrudCommand.java | CRUD handler for sessions (uses new delete API). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/segments/SegmentCrudCommand.java | CRUD handler for segments. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/scoring/ScoringCrudCommand.java | CRUD handler for scoring definitions. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/scopes/ScopeCrudCommand.java | CRUD handler for scopes. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleWatch.java | Repackaged rule watch command into dev.commands. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleTail.java | Repackaged rule tail command into dev.commands. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/rules/RuleStatisticsCrudCommand.java | CRUD handler for rule statistics. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/rules/RuleCrudCommand.java | CRUD handler for rules (includes statistics columns). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RuleResetStats.java | Updated reset-stats command to use shared console base. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/RemoveCommandSupport.java | New shared “remove with confirmation” base command. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/properties/PropertyTypeCrudCommand.java | CRUD handler for property types. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/profiles/ProfileCrudCommand.java | CRUD handler for profiles (plus ID completion). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/profiles/ProfileAliasCrudCommand.java | CRUD handler for profile aliases. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/personas/PersonaCrudCommand.java | CRUD handler for personas. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/ListCommandSupport.java | Repackaged list support and fixed output to session console. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/goals/GoalCrudCommand.java | CRUD handler for goals. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/EventTail.java | Repackaged event tail command and reused shared formatting. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/events/EventCrudCommand.java | CRUD handler for events (uses new delete API). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/DeploymentCommandSupport.java | Refactored deployment support into dev.commands with shared console base. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/DeployDefinition.java | New deploy-definition command under dev.commands. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/consents/ConsentCrudCommand.java | CRUD handler for consents (profile-embedded). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/conditions/ConditionTypeCrudCommand.java | CRUD handler for condition types. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/CommandUtils.java | Shared command utilities (date formatting). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/campaigns/CampaignEventCrudCommand.java | CRUD handler for campaign events. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/campaigns/CampaignCrudCommand.java | CRUD handler for campaigns. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseSimpleCommand.java | Base class for commands needing consistent session console output. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseListCommand.java | New generic list base (shared query/table pattern). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/BaseCommand.java | New generic command base utilities (table + confirm). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/commands/actions/ActionTypeCrudCommand.java | CRUD handler for action types. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/dev/actions/UnomiCrudCommand.java | New unified unomi:crud router command. |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/UndeployDefinition.java | Removed legacy command (replaced by dev.commands). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SessionList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentView.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentRemove.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/SegmentList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/RuleList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ProfileView.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ProfileList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventView.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventSearch.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/EventList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/DeployDefinition.java | Removed legacy command (replaced by dev.commands). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ConditionView.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ConditionList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ActionView.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/src/main/java/org/apache/unomi/shell/commands/ActionList.java | Removed legacy command (replaced by CRUD). |
| tools/shell-dev-commands/pom.xml | Adds commons-csv and adjusts OSGi bundle/plugin metadata. |
| services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java | Implements new ProfileService#deleteSession(). |
| services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java | Implements new EventService#deleteEvent(). |
| bom/pom.xml | Adds an additional managed dependency entry for httpclient-osgi with type bundle. |
| api/src/main/java/org/apache/unomi/api/services/ProfileService.java | Adds ProfileService#deleteSession API. |
| api/src/main/java/org/apache/unomi/api/services/EventService.java | Adds EventService#deleteEvent API. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Run mikepenz/action-junit-report after every integration matrix leg (if: always), with update_check and per-matrix check_name, so a green re-run replaces the stale red JUnit check. Use fail_on_failure=false and require_tests=false; continue-on-error so missing XML cannot fail the job.
…to UNOMI-879-unified-crud-shell
Use identity-based visited sets for circular-reference detection, harden Condition deepCopy and parameter handling, fix Parameter max-depth YAML, wire slf4j.version in unomi-api, and expand unit test coverage.
Fix condition undeploy, rule list performance, null-safety and pagination in CRUD commands, thread-safe consent date formatting, read-only rulestats behavior, segment ID completion, and BOM httpclient dependency management.
Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin so the elasticsearch profile of itests runs ES in a Docker container, mirroring how the opensearch profile already runs OpenSearch. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source. * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that starts/stops a docker.elastic.co/elasticsearch/elasticsearch container, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. This is phase A0 of the PR #757 stack split (see docs/PR-757-stack-extraction-tracker.md). It is intentionally small and self-contained: no test/code changes, only the test infrastructure switch.
…icsearch in integration tests Implements the core configuration switch from UNOMI-921: https://issues.apache.org/jira/browse/UNOMI-921 Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin in the elasticsearch profile of itests, mirroring how the opensearch profile already runs OpenSearch in a Docker container. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source (unchanged from the previous 9400). * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. Heap aligned to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and xpack.security disabled. * Container lifecycle matches OpenSearch exactly: pre-integration-test runs stop+remove then start (with showLogs); post-integration-test runs stop only -- container is kept around for inspection. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. Scope kept minimal for the PR #757 stack split: only the test infrastructure switch lives here. The follow-up UNOMI-921 acceptance items below ship in the platform PR (P) once it lands: * Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in checkSearchEngine() (no longer needed with Docker). * Migrate16xToCurrentVersionIT: replace hardcoded ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort(). * Drop the comments referring to the elasticsearch-maven-plugin template-override workaround. See docs/PR-757-stack-extraction-tracker.md for the full split plan and how this PR fits in the stack.
1 task
1 task
Non-interactive prompts when CI, GITHUB_ACTIONS, or BUILD_NON_INTERACTIVE is set. Add --ci (no Karaf, no Maven build cache) and MAVEN_EXTRA_OPTS for matrix ports.
Replace fixed sleeps in GraphQLListIT and poll after patch refresh in PatchIT.
Poll profile properties after events in PropertiesUpdateActionIT.
Align CI with local developer workflow; pass matrix ports via MAVEN_EXTRA_OPTS.
…icsearch in integration tests Implements the core configuration switch from UNOMI-921: https://issues.apache.org/jira/browse/UNOMI-921 Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin in the elasticsearch profile of itests, mirroring how the opensearch profile already runs OpenSearch in a Docker container. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source (unchanged from the previous 9400). * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. Heap aligned to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and xpack.security disabled. * Container lifecycle matches OpenSearch exactly: pre-integration-test runs stop+remove then start (with showLogs); post-integration-test runs stop only -- container is kept around for inspection. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. Scope kept minimal for the PR #757 stack split: only the test infrastructure switch lives here. The follow-up UNOMI-921 acceptance items below ship in the platform PR (P) once it lands: * Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in checkSearchEngine() (no longer needed with Docker). * Migrate16xToCurrentVersionIT: replace hardcoded ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort(). * Drop the comments referring to the elasticsearch-maven-plugin template-override workaround. See docs/PR-757-stack-extraction-tracker.md for the full split plan and how this PR fits in the stack.
…MI-921-itests-es-docker
# Conflicts: # build.sh # itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java # itests/src/test/java/org/apache/unomi/itests/graphql/GraphQLListIT.java
# Conflicts: # api/src/main/java/org/apache/unomi/api/conditions/Condition.java # api/src/main/java/org/apache/unomi/api/utils/YamlUtils.java # api/src/test/java/org/apache/unomi/api/conditions/ConditionTest.java # build.sh # itests/pom.xml # itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
- Remove duplicate toYaml/deepCopy block in Condition.java (compile error) - Fix OSGi service reference leak in UnomiCrudCommand.findAndExecuteCommand - Fix CSVPrinter closing Karaf session stream (flush instead of close) - Replace new Integer(String) with Integer.parseInt() (removed in Java 17) - Fix paginated read() in CampaignEventCrudCommand and ProfileAliasCrudCommand - Fix silent swallow in ConsentCrudCommand.update() and no-op in Campaign/GoalCrudCommand.update() - Fix ProfileCrudCommand.create() ignoring user-supplied itemId - Fix UndeployDefinition PATCH case returning false success - Add null guards in TailCommandUtils for null timestamps - Fix GraphQLListIT compilation error and assertion message - Minor cleanup: unused imports, Javadoc, vestigial throws declaration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR (merge order)
UNOMI-920-yaml-tostringUNOMI-879-unified-crud-shellThis PR is stacked: merge into
UNOMI-920-yaml-tostringfirst (after #754 is merged tomaster, or equivalent).This branch is stacked: it builds on the UNOMI-920 commit, then adds the UNOMI-879 shell work.
For more information (each PR targets the branch below until the bottom merges): https://github.github.com/gh-stack/introduction/overview/
JIRA
UNOMI-879 — Unified CRUD shell (
shell-dev-commands)The previous Karaf shell commands grew inconsistent across object types (
list/viewstyle varied), duplicated a lot of logic, used different arguments and behaviours for similar operations, and scattered help and docs, which raised maintenance cost and made the shell harder for new operators and contributors. The unifiedunomi:crudstyle (operations × types, shared table/CSV output, JSON create/update, and coherent completion/help) matches UNOMI-879: one predictable pattern, shared plumbing for listing and errors, and less duplicated command code so fixes and features apply everywhere at once.Replaces the older fragmented list/view-style commands with a consistent CRUD-oriented layout under
org.apache.unomi.shell.dev: shared routing (UnomiCrudCommand),CrudCommand/BaseCrudCommand, table/CSV listing support, completers, and type-specific command classes (rules, segments, profiles, events, sessions, definitions, etc.). Deployment / tail / watch helpers are moved into the samedev.commandspackage alongside the new commands.API support (aligned with
unomi-3-devbehaviour):EventService#deleteEvent(String)/EventServiceImpl— remove event by id via persistence.ProfileService#deleteSession(String)/ProfileServiceImpl— remove session by id via persistence.Shell wiring depends on these for
EventCrudCommand/SessionCrudCommanddelete operations.Build:
bom/shell-dev-commandspomupdates as needed for the module.