Error sorting an unsorted result set

[stroomdev10]

Attempt to add sort to page limited results 
java.lang.RuntimeException: Attempt to add sort to page limited results
   at stroom.query.common.v2.CompiledSorters.update(CompiledSorters.java:81)
   at stroom.query.common.v2.LmdbDataStore.fetch(LmdbDataStore.java:782)
   at stroom.query.common.v2.TableResultCreator.create(TableResultCreator.java:121)
   at stroom.query.common.v2.SearchResponseCreator.getResults(SearchResponseCreator.java:256)
   at stroom.query.common.v2.SearchResponseCreator.lambda$create$4(SearchResponseCreator.java:162)
   at stroom.util.logging.LocationAwareLambdaLogger.logDurationIfTraceEnabled(LocationAwareLambdaLogger.java:275)
   at stroom.util.logging.LambdaLogger.logDurationIfTraceEnabled(LambdaLogger.java:80)
   at stroom.query.common.v2.SearchResponseCreator.create(SearchResponseCreator.java:161)
   at stroom.query.common.v2.ResultStore.search(ResultStore.java:255)
   at stroom.query.common.v2.ResultStoreManager.doSearch(ResultStoreManager.java:339)
   at stroom.query.common.v2.ResultStoreManager.lambda$search$1(ResultStoreManager.java:164)
   at stroom.security.impl.SecurityContextImpl.useAsReadResult(SecurityContextImpl.java:405)
   at stroom.query.common.v2.ResultStoreManager.lambda$search$2(ResultStoreManager.java:163)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$2(TaskContextFactoryImpl.java:253)
   at stroom.util.logging.LocationAwareLambdaLogger.logDurationIfDebugEnabled(LocationAwareLambdaLogger.java:307)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$4(TaskContextFactoryImpl.java:253)
   at stroom.util.pipeline.scope.PipelineScopeRunnable.scopeResult(PipelineScopeRunnable.java:36)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$5(TaskContextFactoryImpl.java:250)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$6(TaskContextFactoryImpl.java:262)
   at stroom.security.impl.SecurityContextImpl.useAsReadResult(SecurityContextImpl.java:405)
   at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
   at java.base/java.lang.reflect.Method.invoke(Method.java:580)
   at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50)
   at jdk.proxy2/jdk.proxy2.$Proxy122.useAsReadResult(Unknown Source)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$7(TaskContextFactoryImpl.java:262)
   at stroom.security.impl.SecurityContextImpl.asUserResult(SecurityContextImpl.java:337)
   at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
   at java.base/java.lang.reflect.Method.invoke(Method.java:580)
   at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50)
   at jdk.proxy2/jdk.proxy2.$Proxy122.asUserResult(Unknown Source)
   at stroom.task.impl.TaskContextFactoryImpl.lambda$wrap$11(TaskContextFactoryImpl.java:260)
   at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
   at java.base/java.lang.Thread.run(Thread.java:1583)

[stroomdev66]

This is because a search that does not sort or group data, deliberately terminates as soon as it has at least a page of results. This is an optimisation to prevent excessively long searches in cases where they are not needed, but does result in a partial result set.

In order to accurately group and sort data a search needs to run through all of the data, so searches that are grouped or sorted always require a full result set. Therefore applying a post query sort to a partial result set results in an error.

We could automatically run a new search when sort is applied to a partial result set but in some ways it is no different from trying to change grouping which currently results in the table being cleared awaiting a new search. Perhaps in both cases the table should be cleared although the user might be confused that sometimes they can change the sort order and sometimes it seems they cannot. Perhaps whenever any conditions are changed that would require a new search, the table should show a message or button saying New search required maybe.

Alternatively we could remove the partial result optimisation but it is a useful feature in a resource contended environment. It could however be optional in system config.