Add Azure OpenAI API version environment variable #32
Conversation
| CODEMODDER_AZURE_OPENAI_GPT-3_5-TURBO-2024-12-12_DEPLOYMENT=<DEPLOYMENT_NAME> | ||
| CODEMODDER_AZURE_OPENAI_GPT-4-TURBO-2024-04-12_API_DEPLOYMENT=<DEPLOYMENT_NAME> | ||
| ``` | ||
|
|
||
| - Providing `CODEMODDER_AZURE_OPENAI_API_KEY` without `CODEMODDER_AZURE_OPENAI_ENDPOINT` (and vice versa) will cause a failure on startup. | ||
| - The `CODEMODDER_AZURE_API_VERSION` is optional and codemodders can choose a reasonable default as a fallback. This spec does not intend to be prescriptive about particular fallback versions. |
There was a problem hiding this comment.
What version are we talking about? Are we talking about a "version" of gpt-4, like gpt-4-0613?
There was a problem hiding this comment.
No it's an actual REST API version: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
|
I think this implies that codemodders may be compatible with multiple versions of the Azure OpenAI REST API. That probably means different strategies for each version, since a change in the REST API version usually indicates a breaking change. Do we want to start by only building compatibility for one version of the REST API and later considering whether we need a given codemodder release to be compatible with multiple versions of the REST API? |
I understand your point. l guess the question is whether we want to control this parameter purely within codemodder? Or whether the platform should own it. In either case, I think introducing this to the spec is not problematic. Codemodders should define reasonable defaults so if this variable is not currently defined by the platform there's no problem. |
The platform - or any codemodder user - can use this environment variable to ask codemodder to use REST API version X, but it can't really control whether codemodder supports REST API version X. I think the only thing that the environment variable buys us is the ability to have codemodder support both version X and Y (and Z, etc), such that there's a better chance for compatibility between what codemodder can support and what the caller has available. In any event, it seems like there's some kind of negotiation that needs to happen between codemodder and the platform to figure out if there's a compatible Azure OpenAI REST API available (e.g. another entry in the catalog). |
|
Yeah that's true. Honestly the reason for introducing it is because I assumed this would potentially vary with the endpoint being used but maybe that's not the case. And in any case it doesn't solve the problem of whether the client supports it. Although to be fair this isn't directly codemodder per se as much as whatever dependency it uses for communication with the API. Ultimately though I believe this value is sent as a header in the API requests. |
No description provided.