How to set up OpenAI using Google Gemini API

[y8s]

Is this even possible?
https://ai.google.dev/gemini-api/docs/openai

For example if I plop this in to my compose.yml it fails miserably.

      - OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/
      - OPENAI_API_KEY=FakeAPIKey_$DLKFsfJSd;lkfjSDfuLKSJDfKASJDGFUAQWYf*EWF
      - OPENAI_MODEL=gemini-1.5-flash

mealie-omni-1 | openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'Invalid JSON payload received. Unknown name "response_format": Cannot find field.', 'status': 'INVALID_ARGUMENT', 'details': [{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'description': 'Invalid JSON payload received. Unknown name "response_format": Cannot find field.'}]}]}}]

(Also for a second I thought the "upload an image" was an image of FOOD which would have been amazing if AI could create a recipe from a picture of the food dish.)

Thanks!

[michael-genson]

Seems like they don't support the full API spec yet (which makes sense since it's still in beta).
FYI: https://platform.openai.com/docs/guides/structured-outputs#function-calling-vs-response-format

We haven't switched to the new structured outputs yet (it didn't exist when we implemented the integration).

[y8s]

Ah ok. I was just looking at free alternatives to OpenAI. I also run oobabooga but not regularly enough to warrant pointing to it.

[KiahConnor]

OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
OPENAI_API_KEY=Key generated here https://ai.google.dev/gemini-api/docs/api-key
OPENAI_MODEL=gemini-1.5-flash

This seemed to work intermittently (which is an improvement!), but mainly you just get "error code 500: The model is overloaded. Please try again later". Might have more success with a different model or paid access.

[KiahConnor]

Yeah okay, I think I just had a bad time of day. Did 40 or so images, including handwritten without issue. Glad the above settings helped.

[y8s]

the openai suffix was key!

but I really want the reverse image recipe gen!

[Borriborri]

so do I understand correctly that you guys are using google gemini and not the openai parser? since this is free?

[y8s]

@Borriborri yes, using gemini for free.

[nickmaleao]

I was able to use gemini to import a recipe from an image, but the smart ingredient parsing is failing.

File "/app/mealie/services/parser_services/openai/parser.py", line 95, in _parse
raise Exception("Failed to parse OpenAI response") from e
Exception: Failed to parse OpenAI response

But if I try to use the debug tool it works, it's only failing in the menu when editing a recipe and trying to parse the ingredients using the openai option.