Chat completion with Qwen2.5-instruct model

[GaryTang32]

Hello.

Problem:

I developed a SelectorGroupChat cluster to do some task with tools call.

When I uses GPT4o-mini, it works and able to accomplish the task with tools call.
but when i switch the model into Qwen2.5-instruct model, the cluster failed to use any tools call.

Solution:

After investigation, the problem is Qwen model need a different argument structure when calling chat completion with oai.chat.completions.create. Qwen model's tool_use argument structure is different from GPT4o-mini.

GPT4o-mini:

response = client.chat.completions.create(
    model='',
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "task1"
        }
    ],
    tools={
        'tools': [
            {
                "type": "function",
                "function": {......}
            }
        ],
    }
)

Qwen2.5-instruct:

response = client.chat.completions.create(
    model='',
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "task1"
        }
    ],
    extra_body={
        "extra_body": {
            'tools': [
                {
                    "type": "function",
                    "function": {......}
                }
            ],
        }
    }
)

Qwen model tools_use argument is passed with "extra_body", while GPT is "tools".

And under the "_openai_client.py" in the AutoGen 0.4Dev8. It hardcoded the oai.chat.completion.create using "tools" argument for tools use passing.

therefore when using Qwen model, it wont raise error or exception, but the model unable to decode the tools under "tools" argument, leading unable to use any tools in the cluster.

To resolve the case, changing the oat.chat.completions.create() argument it works.

Update autogen_ext/models/_openai/_openai_client.py

line 460 from:

future = asyncio.ensure_future(
    self._client.chat.completions.create(
        messages=oai_messages,
        stream=False,
        tools=converted_tools,
        **create_args,
    )
)

to:

future = asyncio.ensure_future(
    self._client.chat.completions.create(
        messages=oai_messages,
        stream=False,
        extra_body={
            'extra_body':{
                'tools': converted_tools
            }
        },
        **create_args,
    )
)

this resolve the tools_use argument passing.

Tested this approach with both GPT4o-mini and Qwen2.5-instruct.
Both models able to use tools call after changes.

Is it able to include this change in the next updates? Or is there any method to adopt Qwen model tools call?

Thanks so much.

Best regards,
Gary Tang