Performance, optimizations, and testing

[eckelsjd]

This discussion board is for new testing and performance ideas.

• What is the current memory usage like and where can it be improved? Especially during System.fit() and System.predict(). How do these functions scale memory-wise with increasing number of samples, input dimensions, and the size of field quantities?
• What is the performance hit of constantly calling the imputer during activate_index()? How good is the imputer with the default linear regression settings and should we worry about improving it?
• Analytic models have an extra compress/reconstruct step for field quantities (i.e. when calling to_model_dataset and to_surrogate_dataset inside Component.predict when has_surrogate=False. This may needlessly degrade accuracy for an analytical prediction of a field quantity.
• There seems to be a lot of undefined behavior on how Variable properties/methods like nominal, sample, distribution, and norm behave for field quantities. The only well-defined behavior is returning a list of domains for latent coefficients, but all the other Variable methods are primarily intended for scalars. So what happens when someone tries to use them for a field quantity? For example, what is the PDF of a field quantity? These cases should be tested and better defined.