Support for handling non-existent tools

[roguexz]

It will be nice if we can have custom handling for non-existent tools.

I am playing around with the SystemMessage prompt and tools support. Every now and then the local LLM will hallucinate the tool name, which would result in a runtime error.
e.g,

Tool executor summarizeText not found
java.lang.RuntimeException: Tool executor summarizeText not found
	at dev.langchain4j.internal.Exceptions.runtime(Exceptions.java:32)
	at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:394)
	at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:138)
at DocumentationGenerator$$QuarkusImpl.summarizeDocument(Unknown Source)

Similar to other capabilities within Quarkus, it would be great if there was a way for me to plug in my custom error-handler so that I can remind the LLM about the available list of tools.

e.g.,

 {
    "role" : "tool",
    "content" : "NO SUCH TOOL FOUND: 'summarizeText'.
                 Remember, you only have access to the following tools: {'tool_1', 'tool_2', ...}"
  }

[geoand]

This is definitely an interesting idea.

@cescoffier @edeandrea I wonder if guardrails could be applicable in this case

[cescoffier]

Guardrails for tools is definitely something we want to add, and yes, a "pre-tools-invocation" guardrail would detect that it is trying to invoke something that is not there.

[geoand]

@langchain4j are there any other approaches being discussed in LangChain4j around this topic?

[langchain4j]

Hi @geoand, yes: langchain4j/langchain4j#1052

[geoand]

Thanks @langchain4j!

[edeandrea]

This is definitely an interesting idea.

@cescoffier @edeandrea I wonder if guardrails could be applicable in this case

I think this is definitely a great idea! But I also think that this should be handled in upstream LC4j?

What would be another super idea was a guardrail that could prevent the llm from calling a valid/existing tool when it shouldn't :)

[geoand]

I think this is definitely a great idea! But I also think that this should be handled in upstream LC4j?

Agreed