Directly evaluating a MetadataFilters instance on a BaseNode instance

[ZmeiGorynych]

Hi,
I have an instance of MetadataFilters and a List[BaseNode] and would like to select those nodes from that list that fit the filter. Is there already a function to do that? I'd think there would be a def MetadataFilters.matches(node: BaseNode) -> bool method, but haven't been able to find anything like that.
Reluctant to write my own as it'd have to support arbitrarily deep nested filters, etc.

[dosubot[bot]]

Yes, there is a utility function called filter_nodes that allows you to filter a list of BaseNode instances using a MetadataFilters instance. This function iterates over the nodes and checks if each node's metadata matches the specified filters, adding the matching nodes to the filtered_nodes list. Here's how it is defined:

from llama_index.core.vector_stores import MetadataFilters
from llama_index.core.schema import BaseNode
from typing import List

def filter_nodes(nodes: List[BaseNode], filters: MetadataFilters):
    filtered_nodes = []
    for node in nodes:
        matches = True
        for f in filters.filters:
            if f.key not in node.metadata:
                matches = False
                continue
            if f.value != node.metadata[f.key]:
                matches = False
                continue
        if matches:
            filtered_nodes.append(node)
    return filtered_nodes

This function should help you filter nodes based on the metadata filters you have ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[ZmeiGorynych]

@dosu The function you quote is just a toy implementation from a tutorial, it doesn't handle nested AND and OR filters of arbitrary depth. The other link you give is Qdrant-specific, and doesn't seem to evaluate the filters against a node, but only translate MetadataFilters into a Qdrant-style filter.

[dosubot[bot]]

The LlamaIndex codebase does have a method that handles nested AND and OR filters for MetadataFilters when evaluating against BaseNode instances. The method _parse_filters_recursively is used to convert MetadataFilters into a format that supports nested filters with both AND and OR conditions. This method is part of the OpenSearch integration and is responsible for parsing possibly nested MetadataFilters into an equivalent OpenSearch expression ^[1].

However, if you need to directly evaluate MetadataFilters against BaseNode instances, you might need to implement a custom solution that recursively processes the filters, as the existing utility functions are primarily designed for integration with specific vector stores like OpenSearch or Weaviate.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}