Update ONNX Gen AI jextract example

[ammbra]

1. Status Quo
The current ONNX Gen AI example requires end users to manually build the native library. A lot of files are needed only to build the native runtime, but not serving a need later on.

2. Problem

The native build instructions are subject to change when comes to prerequisites and the build from latest sources can also come with unresolved issues on the native side.

3. Fix

This pull request introduces a few changes in the prerequisites:

• Pre-download compatible releases of onnxruntime and onnxruntime-genai (the onnxruntime-genai is a wrapper over the native onnxruntime). Release archives have the structure:

onnxruntime-genai-0.14.0-osx-arm64/
├── LICENSE
├── README.md
├── SECURITY.md
├── ThirdPartyNotices.txt
├── include
│   ├── ort_genai.h
│   └── ort_genai_c.h
└── lib
   └── libonnxruntime-genai.dylib

onnxruntime-osx-arm64-1.26.0/
├── GIT_COMMIT_ID
├── LICENSE
├── Privacy.md
├── README.md
├── ThirdPartyNotices.txt
├── VERSION_NUMBER
├── include
│   ├── core
│   │   └── providers
│   │       ├── custom_op_context.h
│   │       └── resource.h
│   ├── coreml_provider_factory.h
│   ├── cpu_provider_factory.h
│   ├── onnxruntime_c_api.h
│   ├── onnxruntime_cxx_api.h
│   ├── onnxruntime_cxx_inline.h
│   ├── onnxruntime_env_config_keys.h
│   ├── onnxruntime_ep_c_api.h
│   ├── onnxruntime_ep_device_ep_metadata_keys.h
│   ├── onnxruntime_float16.h
│   ├── onnxruntime_lite_custom_op.h
│   ├── onnxruntime_run_options_config_keys.h
│   ├── onnxruntime_session_options_config_keys.h
│   ├── provider_options.h
│   └── webgpu_provider_factory.h
├── lib
│   ├── cmake
│   │   └── onnxruntime
│   │       ├── onnxruntimeConfig.cmake
│   │       ├── onnxruntimeConfigVersion.cmake
│   │       ├── onnxruntimeTargets-release.cmake
│   │       └── onnxruntimeTargets.cmake
│   ├── libonnxruntime.1.26.0.dylib
│   ├── libonnxruntime.1.26.0.dylib.dSYM
│   │   └── Contents
│   │       ├── Info.plist
│   │       └── Resources
│   │           ├── DWARF
│   │           │   └── libonnxruntime.1.26.0.dylib
│   │           └── Relocations
│   │               └── aarch64
│   │                   └── libonnxruntime.1.26.0.dylib.yml
│   ├── libonnxruntime.1.dylib -> libonnxruntime.1.26.0.dylib
│   ├── libonnxruntime.dylib
│   └── pkgconfig
│       └── libonnxruntime.pc
└── testdata
   └── libcustom_op_library.dylib

The user should manually move the libonnxruntime<release>.dylib in the onnxruntime-genai lib folder.

• Generate Java bindings for the downloaded onnxruntime-genai release using the
• Removes unnecessary variables in prompt method code and add a way to exit the from LLM conversation in call method.

I run tests through Github actions enabled in my fork.

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Change must be properly reviewed (no review required)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jextract.git pull/304/head:pull/304
$ git checkout pull/304

Update a local copy of the PR:
$ git checkout pull/304
$ git pull https://git.openjdk.org/jextract.git pull/304/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 304

View PR using the GUI difftool:
$ git pr show -t 304

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jextract/pull/304.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back amihalceanu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@ammbra This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

Update ONNX Gen AI jextract example

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 1 new commit pushed to the master branch:

• 791f3c3: Fix gh actions

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[mlbridge]

Webrevs

• 01: Full - Incremental (e10023d9)
• 00: Full (43a46d13)

[mcimadamore]

I can't find a lot of references to this variable -- could we please double check this is really needed?

[mcimadamore]

E.g. our compile.sh script doesn't need this -- so who does? Is it jextract, or is the execution of the Java program? If the latter, it feels like there should be better options to point FFM to the correct libraries

[mcimadamore]

I see -- that part is depended on by run.sh, not compile.sh.

I'd suggest to reorder the bullets:

1. set ORT_GENAI_INCLUDE_DIR
2. run compile.sh
3. download model
4. set ONNX_LIB_PATH
5. run run.sh

And, I think it would probably be even better to point jextract at the two libraries explicitly using -l flags, so that step (4) can be omitted entirely.

[ammbra]

I reordered and reworded the steps, but I think is easier to run the example when the native library is given at runtime.

[mcimadamore]

I guess what I'm trying to get at is a place where maybe we don't need to ask the user to copy a library from one place to another? If ONNX_LIB is used to set java.library.path (which allows multiple paths) perhaps it would be better to ask developers to set two variables:

1. ONNX_HOME
2. ONNX_GENAI_HOME

And then derive from (2) the include folder for jextract and from (1 + 2) the correct -Djava.library.path -- e.g. as in
-Djava.library.path=$ONNX_HOME/lib:$ONNX_GENAI_HOME/lib ?