[Question]: Pydantic models in function tool arguments

[ziodave]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I have the following function:

class AClass(BaseModel):
    value: str = Field(..., description="A value")

def my_function_tool(
        list_of_something: list[str] = Field(
            description="A list of something"
        ),
        list_of_classes: list[AClass] = Field(
            description="A list of classes"
        )
):

I connect it to the agent:

    tool = FunctionTool.from_defaults(
        my_function_tool
    )

    agent = OpenAIAgent.from_tools(
        llm=llm,
        tools=[tool],
        verbose=True
    )

However when the function is called, it passing a list[dict] to list_of_classes, instead of list[AClass].

Shouldn't LlamaIndex reify the dicts to AClass instances?

[dosubot]

LlamaIndex does not automatically reify lists of dictionaries to lists of Pydantic model instances. When using Pydantic models in function tool arguments, you need to handle the conversion manually. The framework provides methods like parse_obj and parse_raw for converting dictionaries or raw JSON strings into Pydantic model instances, but this is typically done for individual objects rather than lists.

To achieve the desired behavior, you can manually convert the list of dictionaries to a list of AClass instances within your function. Here's an example of how you might modify your function to handle this:

from pydantic import BaseModel, Field

class AClass(BaseModel):
    value: str = Field(..., description="A value")

def my_function_tool(
        list_of_something: list[str] = Field(
            description="A list of something"
        ),
        list_of_classes: list[dict] = Field(
            description="A list of classes"
        )
):
    # Convert list of dicts to list of AClass instances
    list_of_classes = [AClass.parse_obj(item) for item in list_of_classes]
    # Now you can work with list_of_classes as a list of AClass instances

This approach ensures that each dictionary in list_of_classes is converted to an AClass instance before you use it in your function.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[ziodave]

@dosu in this case how do I define the properties that map to openapi?

[dosubot]

To define the properties that map to OpenAPI when using Pydantic models with function tool arguments, you can use the get_parameters_dict method from the ToolMetadata class. This method generates a dictionary representing the OpenAPI schema for the function's parameters. If a custom fn_schema is provided, it uses the model_json_schema method from the Pydantic BaseModel to generate the schema. Otherwise, it defaults to a predefined schema.

Here's a code snippet illustrating this:

def get_parameters_dict(self) -> dict:
    if self.fn_schema is None:
        parameters = {
            "type": "object",
            "properties": {
                "input": {"title": "input query string", "type": "string"},
            },
            "required": ["input"],
        }
    else:
        parameters = self.fn_schema.model_json_schema()
        parameters = {
            k: v
            for k, v in parameters.items()
            if k in ["type", "properties", "required", "definitions", "$defs"]
        }
    return parameters

This method ensures that the properties are correctly mapped to the OpenAPI format by extracting the necessary fields from the Pydantic model's JSON schema. It also supports nested Pydantic models, ensuring that nested structures are correctly represented in the OpenAPI schema using $defs ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[ziodave]

@logan-markewich I don't understand, isn't LlamaIndex supposed to reify the function args?

[logan-markewich]

@ziodave llms can only write string, dict, list, int, and float types. It passes that into the function. Whatever happens after that is up to the function to figure out.

In this case, I would just check for dict and convert it

There's maybe a way to do type introspection to get the class object and cast the input for you, maybe a PR worth looking into

[ziodave]

Thanks @logan-markewich

In the meanwhile this is what I am doing, if it helps others:

def _reify_argument(my_list: list[Union[dict | MyClass]]) -> list[MyClass]:
    return [MyClass(**m) if isinstance(m, dict) else m for m in my_list]


def function_tool(my_list: list[MyClass] = Field(description="A list of My Classes")):

    my_list = _reify_argument(my_list)