Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assistant is thrown into infinite loop if assistant.tool_choice = "function_name" is selected and autorun. #825

Open
andreibondarev opened this issue Oct 12, 2024 · 0 comments
Labels
assistants Related to Langchain::Assistant class bug Something isn't working thoughtbot-open-summit-2024 https://thoughtbot.com/events/open-summit

Comments

@andreibondarev
Copy link
Collaborator

andreibondarev commented Oct 12, 2024

Describe the bug
The Assistant is thrown into an infinite loop when LLM is forced to pick the same tool on every request. Since we auto-run the tool calls and then call the LLM again -- it's thrown into an infinite loop.

To Reproduce

llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])
assistant = Langchain::Assistant.new(
  llm: llm,
  instructions: "You're a helpful AI assistant",
  tools: [Langchain::Tool::NewsRetriever.new(api_key: ENV["NEWS_API_KEY"])],
  tool_choice: "..."
)
assistant.add_message_and_run!(content: "What's the latest news about AI?")

Observe the Assistant thrown into an infinite loop because:

  1. It calls the LLM which is forced to return a tool call
  2. Tool call is auto-run
  3. It calls the LLM again which is forced to return a tool call again
  4. And it keeps repeating

Expected behavior
The openai/swarm framework allows setting max_turns: which (I believe?) stops this from happening. We may consider something similar.

@andreibondarev andreibondarev added bug Something isn't working assistants Related to Langchain::Assistant class labels Oct 12, 2024
@andreibondarev andreibondarev added the thoughtbot-open-summit-2024 https://thoughtbot.com/events/open-summit label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assistants Related to Langchain::Assistant class bug Something isn't working thoughtbot-open-summit-2024 https://thoughtbot.com/events/open-summit
Projects
None yet
Development

No branches or pull requests

1 participant