Reduce memory footprint

[mortonjt]

The current memory footprint is quite high - the batch size maxes out at 32.
We could significantly speed things up if we could reduce the memory footprint.
One possibility is to utilize more sparse matrices in the internal computations -- particularly for the masking and alignment matrices.
See https://stackoverflow.com/a/56887077/1167475