[Feature] ChatQnA helm chart should support remote inference endpoint

[yongfengdu]

Priority

P2-High

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

This PR introduced new manifests file and README to support remote inference endpoint, but this is not supported by helm chart deployment yet.
opea-project/GenAIExamples#1149
We should support this feature as it's common for user to choose a cloud provided endpoint.

[yongfengdu]

First thing is ChatQnA should add support for endpoints configuration - LLM_SERVER_HOST_IP, LLM_SERVER_PORT, RERANK_SERVER_HOST_IP, RERANK_SERVER_PORT, EMBEDDING_SERVER_HOST_IP, EMBEDDING_SERVER_PORT which are defined at https://github.com/opea-project/GenAIInfra/blob/main/helm-charts/chatqna/templates/deployment.yaml#L37
So far, they are either hard coded, or auto spell by different backend.

Another thing is to add envs for passing TOKEN. In the above commit, they are CLIENTID, CLIENT_SECRET and TOKEN_URL for Oauth.
I'm not familiar with this and haven't figure out how the above 3 envs will be passed to the specified llm endpoint from the code.

Also we should consider the following:
Security way to protect the SECRET.
Other authentication method?

[yongfengdu]

Depends on this one.
chatqna is not using llm-uservice, so these ENVs will not be passed down to llm_endpoint.
opea-project/GenAIExamples#1453