Making AI interactions more robust

When implementing an AI service, leveraging Quarkus fault tolerance annotations can enhance the reliability of your interactions. To integrate fault tolerance, follow these steps:

  • 1. Start by adding the quarkus-smallrye-fault-tolerance dependency to your pom.xml file:

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-smallrye-fault-tolerance</artifactId>
</dependency>

  • 2. Utilize the @Fallback or @Timeout annotation within the AIService class. These annotations help manage exceptional scenarios:

package io.quarkiverse.langchain4j.samples;

import java.time.temporal.ChronoUnit;

import org.eclipse.microprofile.faulttolerance.Fallback;
import org.eclipse.microprofile.faulttolerance.Timeout;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface AiServiceWithFaultTolerance {

    @SystemMessage("You are a professional poet")
    @UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
    @Timeout(value = 60, unit = ChronoUnit.SECONDS)
    @Fallback(fallbackMethod = "fallback")
    String writeAPoem(String topic, int lines);

    default String fallback(String topic, int lines) {
        return "I'm sorry, I can't write a poem about " + topic;
    }
}

A Note on Utilizing Tools

When employing tools in your interactions, multiple requests and responses occur in the background, potentially extending the time required to obtain a response from the AI service. It’s essential to consider this while configuring a timeout to ensure robustness in your system’s interactions.