Tool Insights

[kichanyurd]

Motivation

This discussion is based on issues raised in #237 by @ElisonSherton. Primarily:

We need to show the message generator why certain tools were not triggered in order for it to not infer or assume things that it doesn't know.

Consider the following use case:

1. We have the following guideline: When the user needs to make a transaction; Then assist them in the process connected to a tool transfer_money(destination_account: str, amount: float, payment_date: str).
2. A customer sends a message, Hey, I'd like to send $200 to John's account
3. Note how the customer didn't specify a payment date. Since payment_date is a required parameter, the tool cannot be called. The ToolCaller understands this and therefore does not attempt to call transfer_money().
4. Today, however, the MessageEventGenerator has no way to know that the ToolCaller was only missing one argument to make the call.
5. As a result, the MessageEventGenerator, not seeing a success indicator from a tool call event, would still try to comply with the its guideline to assist them in the process (of making a transaction).
6. This is likely to result in a generated message such as, I would love to help you with this transaction. Let me see what I can do and get back to you. or something fairly useless of that nature.
7. Note that it wouldn't actually do anything as a follow up... that response is just a fallback to it not seeing any clear way to help, while it also doesn't understand what else it could possibly need from the user (and it tries hard not to hallucinate some random need).
8. So, in essence, the MessageEventGenerator is not to blame here. It just lacks information, which the ToolCaller has; namely, that it would have liked to run the transfer_money() tool — it just couldn't because payment_date was missing.

Solution Expectations

1. We need to bring the ToolCaller's evaluation into the MessageEventGenerator's context, making it accessible for its consideration.
2. It would also be useful to be able to say, per parameter, where do we expect to get it from:
   1. From the customer?
   2. From the context?
   3. Make something up? :)
3. While we're at it, if we're improving the ability to annotate parameters, we should consider supporting parameter types which we current don't; e.g., datetime. We currently only support basic types like str, int, etc.

Solution Proposal

Part 1: Communicating Tool Insights to the MessageEventGenerator

TBD

Part 2: Supporting Additional Parameter Types

TBD

Part 3: Improving the Ability to Annotate Parameters

TBD

Discussion

I will update the Solution Proposal soon enough. I'd like to first get some input on the above and see if anyone has additional interesting use cases for consideration here.