Azure Openai still doesn't work

[robertobalestri]

Using what's told in the Docs, and also tried everything else. It's month that it is not working.

AZURE_OPENAI_API_KEY=[Your Key]
AZURE_OPENAI_ENDPOINT=https://{your-endpoint}.openai.azure.com/
OPENAI_API_VERSION=2024-05-01-preview

note that the deployment name must be the same as the model name

FAST_LLM=azure_openai:gpt-4o-mini
SMART_LLM=azure_openai:gpt-4o
STRATEGIC_LLM=azure_openai:o1-preview

EMBEDDING="azure_openai:text-embedding-ada-002"

[ElishaKay]

Welcome @robertobalestri,

Great persistence!

Can you try this and let us know if it works:

AZURE_OPENAI_API_KEY=xxxxxx
AZURE_OPENAI_ENDPOINT=https://xxxxxxx.openai.azure.com/openai/
AZURE_OPENAI_API_VERSION=2024-08-01-preview


FAST_LLM=azure_openai:gpt-4o-mini
SMART_LLM=azure_openai:gpt-4o
STRATEGIC_LLM=azure_openai:gpt-4o

The model name and deployment name must be similar, ie gpt-4o for gpt-4o model.

This user in the discord says: "I actually looked at all tickets that mentioned azure_openai and tried all the different variations of .env, and finally noticed that there was /openai/ at the end of the base url for most of them who made it work.
I don't think it is isolated to my use case as I used a vanilla fresh install."

If it works for you, you can go for the PR.

[robertobalestri]

I still got the problem, it's exactly what I already pass to the program. Sending it without OPENAI_API_VERSION it gives a pydantic error because it miss a field OPENAI_API_VERSION, so i decided to try to circumnavigate this adding both API VERSION:

AZURE_OPENAI_API_KEY=37f05a9d-696e-4c92-8651-31e76ffa6750
AZURE_OPENAI_ENDPOINT=https://nead3.openai.azure.com/
AZURE_OPENAI_API_VERSION=2024-08-01-preview
OPENAI_API_VERSION=2024-08-01-preview

[ElishaKay]

What error do you get back when you run this test script:

Test your LLM

That test script is probably the fastest way to iterate towards a solution - since it leverages just the underlying chat api helper function of GPTR

[Aadhithya-D]

I created the same .env file as @robertobalestri (with AZURE_OPENAI_API_VERSION=2024-08-01-preview and OPENAI_API_VERSION=2024-08-01-preview). While this resolved the previous issue, I am now encountering an embedding rate limit, which is preventing further progress. Could you please provide guidance on how to proceed? Thank you!

[ElishaKay]

add an env var EMBEDDING_PROVIDER=azure_openai

If that doesnt work try reverting back to version 0.10.3 of the gptr pip package

[danieldekay]

It is working for me, as described in #1061 (comment)

[ElishaKay]

@danieldekay 's fix is now in the docs