Does Autogen 0.4 support streaming mode?

[easonktsai]

I noticed that in the class BaseOpenAIChatCompletionClient, the 'stream' parameter in self._client.chat.completions.create is hardcoded to 'False'. Does this mean that streaming is not currently supported?

[ekzhu]

Looks down a bit and you will find create_stream

[easonktsai]

How can I enable streaming responses for the AzureOpenAIChatCompletionClient in RoundRobinGroupChat?

from autogen_ext.models.openai import AzureOpenAIChatCompletionClient

model_client = AzureOpenAIChatCompletionClient(
            azure_endpoint="",
            api_version="2024-08-01-preview",
            api_key="",
            model='gpt-4o',
        )

async def get_weather(location: str) -> str:
    return f"The weather in {location} is sunny."

assistant = AssistantAgent(
    "Assistant",
    model_client=model_client,
    tools=[get_weather],
)
termination = TextMentionTermination("TERMINATE")
team = RoundRobinGroupChat([assistant], termination_condition=termination)
await Console(team.run_stream(task="What's the weather in New York?"))

[ekzhu]

Not available at the moment. Please follow the development: #3862 . Please comment on the issue if the streaming usage case is critical for you.