Ability to rewrite LLM input

[Ziaunys]

Is there currently any way to rewrite the LLM input using this extension? I see that with the OutputGuardrails you can do this by using successWith(). However, there doesn't appear to be a similar API for InputGuardrails. Is there another way to do this outside of just sanitizing the input before passing it to the LLM? It's fine if I have to do that.

The problem I'm running into is that if I don't sanitize the inputs myself then users can pass in {} characters which get interpolated by Qute templates. I'd like to avoid this.

[geoand]

cc @cescoffier

[cescoffier]

@mariofusco added this a few months ago. It should be possible to write an input guardrail that modify the user input.

[Ziaunys]

Oh, excellent. I happened to be looking at an older version. Thank you! Exactly what I wanted. 🙏

[Ziaunys]

I just updated the extension and tried using successWith(String rewrittenInput) on the validate(String userMessage) method, however, it looks like the Qute template evaluation happens before the guardrails run which results in a template exception thrown if there is any user input like:

Hello {bar} This will throw an exception

Exception:

: io.quarkus.qute.TemplateException: Rendering error: Key "bar" not found in the template data map with keys [current_date, response_schema, memoryId, current_time, current_date_time] in expression {bar}
        at io.quarkus.qute.TemplateException$Builder.build(TemplateException.java:169)
        at io.quarkus.qute.EvaluatorImpl.propertyNotFound(EvaluatorImpl.java:234)
        at io.quarkus.qute.EvaluatorImpl.resolve(EvaluatorImpl.java:204)
        at io.quarkus.qute.EvaluatorImpl.resolveReference(EvaluatorImpl.java:131)
        at io.quarkus.qute.EvaluatorImpl.evaluate(EvaluatorImpl.java:85)
        at io.quarkus.qute.ResolutionContextImpl.evaluate(ResolutionContextImpl.java:29)
        at io.quarkus.qute.ExpressionNode.resolve(ExpressionNode.java:36)
        at io.quarkus.qute.Results.resolveWith(Results.java:107)
        at io.quarkus.qute.Results.resolveAndProcess(Results.java:57)
        at io.quarkus.qute.SectionNode$SectionResolutionContextImpl.execute(SectionNode.java:238)
        at io.quarkus.qute.SectionHelper$SectionResolutionContext.execute(SectionHelper.java:66)
        at io.quarkus.qute.Parser$1.resolve(Parser.java:1331)
        at io.quarkus.qute.SectionNode.resolve(SectionNode.java:53)
        at io.quarkus.qute.SectionNode.resolve(SectionNode.java:58)
        at io.quarkus.qute.TemplateImpl$TemplateInstanceImpl.renderData(TemplateImpl.java:272)
        at io.quarkus.qute.TemplateImpl$TemplateInstanceImpl.renderAsyncNoTimeout(TemplateImpl.java:239)
        at io.quarkus.qute.TemplateImpl$TemplateInstanceImpl.render(TemplateImpl.java:164)
        at io.quarkiverse.langchain4j.QuarkusPromptTemplateFactory$QuteTemplate.render(QuarkusPromptTemplateFactory.java:69)
        at dev.langchain4j.model.input.PromptTemplate.apply(PromptTemplate.java:102)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.prepareUserMessage(AiServiceMethodImplementationSupport.java:675)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:162)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:138)
        at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$1.apply(MethodImplementationSupportProducer.java:31)
        at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$1.apply(MethodImplementationSupportProducer.java:28)
        at io.quarkiverse.langchain4j.runtime.aiservice.SpanWrapper.wrap(SpanWrapper.java:32)
        at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$2.apply(MethodImplementationSupportProducer.java:40)
        at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$2.apply(MethodImplementationSupportProducer.java:37)
        at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1.implement(MethodImplementationSupportProducer.java:46)
        at io.smallrye.context.impl.wrappers.SlowContextualRunnable.run(SlowContextualRunnable.java:19)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1848)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:627)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
        at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1575)

[Ziaunys]

Another update. I was able to work around this issue by setting the following properties:

quarkus.qute.strict-rendering=false
quarkus.qute.property-not-found-strategy=output-original

This is maybe not the best solution but at least an exception won't be thrown and I can handle the input in guardrails.