UserProxyAgent sends empty message

[quarechen]

Describe the bug

when user_proxy use NEVER hunman input mode, in the second round of dialogue，user_proxy send empty message to assistant：

Steps to reproduce

the code is agentchat_teaching.ipynb:

import autogen

llm_config = {
    "timeout": 600,
    "cache_seed": 50,  # change the seed for different trials
    "config_list": autogen.config_list_from_json(
        "OAI_CONFIG_LIST",
        filter_dict={"model": ["GLM-4"]},
    ),
    # "temperature": 0,
}

# create an AssistantAgent instance named "assistant"
assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config=llm_config,
    is_termination_msg=lambda x: True if "TERMINATE" in x.get("content") else False,
)
# create a UserProxyAgent instance named "user_proxy"
user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    is_termination_msg=lambda x: True if "TERMINATE" in x.get("content") else False,
    max_consecutive_auto_reply=10,
    code_execution_config={"work_dir": "coding", "use_docker": True}
)

task1 = """
Find arxiv papers that show how are people studying trust calibration in AI based systems.
"""

user_proxy.initiate_chat(assistant, message=task1)

Model Used

GLM-4

Expected Behavior

I think maybe when message is empty, the agent should stop,or return some prompt to the caller?

Screenshots and logs

the result is:

user_proxy (to assistant):


Find arxiv papers that show how are people studying trust calibration in AI based systems.


--------------------------------------------------------------------------------
assistant (to user_proxy):

To find arXiv papers on the topic of trust calibration in AI-based systems, we can use the arXiv API to perform a search. Since the user can only execute code and not modify it, I will provide a Python script that searches arXiv for relevant papers and prints out their titles and abstracts.

Here's the plan:
1. Use the arXiv API to perform a search for papers related to trust calibration in AI-based systems.
2. Parse the results to extract paper titles and abstracts.
3. Print out the results.

Below is the Python script that accomplishes this:

// python code omit ...

To execute this script, the user should save it as `search_arxiv_papers.py` and run it with Python 3. The script will output the titles and abstracts of the papers found.

Please execute the script and provide the output, or let me know if there are any issues. If the script doesn't return relevant results, we might need to refine the search terms or explore additional APIs or databases.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output: 
Title: Trust Calibration and Trust Respect: A Method for Building Team Cohesion
  in Human Robot Teams

// 10 list of papers Omit ...
---


--------------------------------------------------------------------------------
assistant (to user_proxy):

The search results from the arXiv papers provide a comprehensive overview of the research being conducted on trust calibration in AI-based systems. Here's a summary of the key points from the abstracts:

1. **Trust Calibration and Respect in Human-Robot Teams**: This paper discusses the importance of trust calibration in human-robot interaction (HRI) and introduces the concept of "trust respect," where a robot assesses whether a human's mistrust is due to external factors and respects the trust level accordingly.

2. **Clustering Human Trust Dynamics**: The paper proposes a methodology to develop customized trust models by clustering humans based on their trust dynamics, which improves trust prediction performance for trust calibration.

// Omit...

10. **Trust-Aware Decision Making for Human-Robot Collaboration**: A computational model is introduced that integrates trust into robot decision-making, using a trust-POMDP model to infer human trust and maximize team performance.

These papers cover various aspects of trust calibration in AI-based systems, including human-robot interaction, automated driving, and human-AI collaboration. The research highlights the importance of understanding and modeling trust dynamics, as well as the development of adaptive and real-time trust calibration techniques.

--------------------------------------------------------------------------------
user_proxy (to assistant):



--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspaces/autogen/test/mytest.py", line 32, in <module>
    user_proxy.initiate_chat(assistant, message=task1)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 985, in initiate_chat
    self.send(msg2send, recipient, silent=silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 627, in send
    recipient.receive(message, self, request_reply, silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 788, in receive
    self.send(reply, sender, silent=silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 627, in send
    recipient.receive(message, self, request_reply, silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 788, in receive
    self.send(reply, sender, silent=silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 627, in send
    recipient.receive(message, self, request_reply, silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 788, in receive
    self.send(reply, sender, silent=silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 627, in send
    recipient.receive(message, self, request_reply, silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 788, in receive
    self.send(reply, sender, silent=silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 627, in send
    recipient.receive(message, self, request_reply, silent)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 786, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 1890, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 1273, in generate_oai_reply
    extracted_response = self._generate_oai_reply_from_client(
  File "/workspaces/autogen/autogen/agentchat/conversable_agent.py", line 1292, in _generate_oai_reply_from_client
    response = llm_client.create(
  File "/workspaces/autogen/autogen/oai/client.py", line 623, in create
    response = client.create(params)
  File "/workspaces/autogen/autogen/oai/client.py", line 276, in create
    response = completions.create(**params)
  File "/home/vscode/.local/lib/python3.10/site-packages/openai/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
  File "/home/vscode/.local/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 667, in create
    return self._post(
  File "/home/vscode/.local/lib/python3.10/site-packages/openai/_base_client.py", line 1208, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/home/vscode/.local/lib/python3.10/site-packages/openai/_base_client.py", line 897, in request
    return self._request(
  File "/home/vscode/.local/lib/python3.10/site-packages/openai/_base_client.py", line 988, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'code': '1214', 'message': 'messages[5]:content和tool_calls 字段不能同时为空'}}

Additional Information

the env is docker

[ekzhu]

Thanks for the issue. Looks like you can set a default reply message in the user proxy: see default_auto_reply https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent

[afourney]

If you would like it to terminate when default_auto_reply is reached, you can set the default_auto_reply to "TERMINATE", or set is_termination_message on the recipient to check for empty messages. If you would like the agent to call the LLM, add an llm_config to user proxy.