Support the Jina AI jina-embeddings-v3 model

[davidkyle]

When trying to import the jina-embeddings-v3 model I encountered the error described in https://huggingface.co/jinaai/jina-embeddings-v3/discussions/96. The stated fix on the discussion is to install the flash-attn package.

Support may depend on resolving #721 first

[baerchen110]

+1

[wsosnowski]

+1

[mahytom]

+1

[david-fennessey]

+1

[baerchen110]

@davidkyle do we have an updates since we acquired Jina?

[Danouchka]

+1

[arisonl]

We don’t currently have plans to add JinaAI support to Eland. The direction is that Elastic's models will be delivered via EIS. CPU-based ML nodes are are not a future-proof infrastructure path, and the deployment experience doesn’t align with where we’re taking model delivery and management. Self-managed and hybrid deployments will be able to use Elastic models on EIS via Cloud Connected mode.

[Danouchka]

Hi @arisonl there are gov organisations or institutions that are strictly forbidden to send anything into the cloud, outside their private datacenter as you know. How to cope with those use cases please ?

[Danouchka]

I think we should be able to serve its own models via ML nodes as public institutions that have strong uses cases on low volumetry but high value data. They dont need to scale , they just need a ML node to do the job.
Sometimes, it can be confidential or secret data for which they want embeddings......

[mahytom]

@Danouchka i fully agree with you . In our institution we are not allowed to send data outside of our network, and it will only get worse with sovereignty requirements coming in . One of the big reasons to get an elastic license is because of all the build in vector features . If we need to externalize this to our own models and custom layer to pre process data then we might as well use the open source version of elastic .
ML nodes can run on GPU bases instances and should be able to make use of the computing available.

[gregoireco]

Now that jina-v5 is out, if any effort should go into supporting jina models, I would suggest to consider jumping directly to v5.
It's multilingual, the nano variant is smaller and yet beats v3 relevance wise.

[UweW]

There are some use cases where a cloud-hosted model is not an option.
These include ring-fenced environments, as well as lab and demo scenarios. Or very smal environments.
It also doesn’t always make sense to run models on GPUs, even though we’ve been waiting a long time for this feature. Especially when only demo data is being loaded, or the dataset is quite small, there’s no need for expensive GPUs or cloud access. Or when you want to compare models under identical conditions.
Since the restrictions also apply to Elastic's own model (jinaai) as well as the new v5 model, this makes it difficult for customers to understand