Running embedding in batch

[michael-shaked]

Hi @kherud ,

First of all, thank you for your great project!

I'm trying to embed strings using the library, but I'm facing difficulties when trying to process multiple strings in batches. The current implementation works with single record, but I need assistance in how to modify the code to handle embedding of multiple strings at once, i.e., in batches.

Regards,
Michael

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

Resolved in the downstream fork at https://github.com/bernardladenthin/java-llama.cpp.
In addition to the single-prompt LlamaModel.embed(String), the fork exposes
handleEmbeddings(String paramsJson, boolean oaiCompat)
(LlamaModel.java:316); the native helper extract_embedding_prompt
(src/main/cpp/json_helpers.hpp:137) accepts either a string or an array of
strings, enabling batched embeddings.