How do you limit on an aggregation function output?

[Avi-D-coder]

What is the correct way to write this script?
I'm assuming the cozo does not know sum is strictly increasing?

nums[i, a] <- [[1, 1], [2, 1], [3, 3], [4, 4], [5, 5]]
ag[a, count(a), sum(a)] := nums[i, a]
?[a, count, sum] := ag[a, count, sum], sum <= 3

[zh217]

I don't think I understand the problem very well, the script is working as far as I can see.

[Avi-D-coder]

nums[i, a] := *nums{i, a}

If instead of a constant rule nums was a large stored relation (maybe with an index on {a, i}), would every row of the stored relation nums be read and summed or would only [1, 1], [2, 1], [3, 3] be read. I don't yet fully grok how scripts evaluate.

[Avi-D-coder]

Here is the explain output. Having an index on nums{a} does not seem to matter. Am I correct in thinking the empty filter in row 3 op load_stored) means the whole nums relation is read into memory?

[zh217]

I think you are trying to count and sum numbers less than three only. In this case, your program does not give you want you want. You want to write the following instead:

nums[i, a] <- [[1, 1], [2, 1], [3, 3], [4, 4], [5, 5]]
ag[a, count(a), sum(a)] := nums[i, a], a <= 3
?[a, count, sum] := ag[a, count, sum]

On whether everything will be read into memory: it's not straightforward to explain. If all aggregation operators are semi-lattice operators, and the body can operate by streaming, then there is no useless build-up of intermediate relations in memory.

[Avi-D-coder]

My goal is to filter on sum <= 3 not a, but I think your answer clarifies evaluation. Thank you.

Let me see if I understand?

ag[a, count(a), sum(a)] := nums[i, a]
?[a, count, sum] := ag[a, count, sum], sum <= 3

So because count and sum are not semi-lattice aggregations ag cannot operate by streaming. ag is fully built up in memory.
Then the filter is applied to ag to build ? which is returned. Or is ? built incrementally at the same time ag is built?

[zh217]

Anything with aggregations in it acts as points of stratification, so if anything else depends on them, they must be fully built in memory.

But with the above example, if nums were stored instead of inline, then in this case nums can be streamed and do not need to be fully in memory.