Advice on keeping attributes up to date after slicing

[matthew-brett]

Could I ask for advice about attributes and array slicing?

My particular case is that I'm representing functional and structural MRI images as Xarrays - see e.g. this notebook.

This has many advantages, but one complexity is that each 3D brain volume has an associated [affine])https://nipy.org/nibabel/coordinate_systems.html) - a 4x4 array that maps coordinates in voxel space to coordinates in the real world - for example in terms of millimeters from the center of the scanner.

The transformation is conceptually fairly simple, but for now, lets just note that it includes the real-world coordinate of the voxel 0, 0, 0 in the array. Call this the origin part of the affine.

At the moment I store the affine as a 4x4 array attribute.

The problem is that, when I slice the 3D array - e.g. img[:, :, 10:20], the corresponding affine needs to change to reflect the fact that the voxel at current coordinate 0, 0, 0 is the voxel that used to be at coordinate 10, 0, 0, and therefore its real-world coordinate in origin has to change accordingly.

I'm not sure how to achieve that with Xarray. I could try and divert users to do all slicing operations via an accessor, and wrap them accordingly to fix up the affine, but that seems very error-prone - I guess many people will nevertheless use standard indexing, and evade my attempts to keep track of the affine. Is there a better way to add hooks to the indexing interfaces to apply the logic I need to keep the affine up to date?

[effigies]

Could be worth taking a page out of FreeSurfer's book and decompose the affine into the cosines, zooms and translations. The cosine could be static metadata on the axis, while the zooms and translations could be incorporated into the axis ticks, such that img[:, :, 10:] increases the translation by 10 ticks and img[:, :, ::2] doubles the tick spacing. The origin should be recoverable by looking at the location of img[0, 0, 0].

I say this without a definite model in my head of xarray's capabilities.

[matthew-brett]

Aha - yes - I hadn't thought of putting the origin into the axis ticks - nice idea - let me reflect.

[matthew-brett]

Reflecting - I think we can get a long way by just putting the i, j, k range(0, len(img['i']) coordinates on the spatial axes. In that way, after slicing, we have the current voxel coordinate in the original voxel block, and the original origin, and we can calculate the new origin.

We could also do a version of what you're suggesting - but I think that would be a bit more complicated. Let's write the affine matrix as:

$$ \left[ {\begin{array}{cc} a_{1,1} & a_{1,2} & a_{1,3} & o_x \\ a_{2,1} & a_{2,2} & a_{2,3} & o_y \\ a_{3,1} & a_{3,2} & a_{3,3} & o_z \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$

where $o_x, o_y, o_z$ are the original origin values.

After slicing with say img[10:, :, :] the new origin should be $a_{1,1} * 10 + o_x, a_{2,1} * 10 + o_y, a_{2,1} * 10 + o_z$. So for the general case where $a_{2,1}$ or $a_{3,1}$ need not be 0, if we wanted to read the new origin off from the 'i' axis labels, we'd need tick labels with three coordinates - do you agree?

[effigies]

Agreed. Was thinking that oblique or shear affines would break my proposal. I think just tracking the indices on the axis would be the cleanest way forward.

[matthew-brett]

Sorry - to add - Chris' suggestion above is a workaround; using that workaround, we can work out what slicing the user has done to the object, and adjust the object accordingly - but that still means that, after the user has sliced the object, the object is in an unresolved state, where the metadata no longer matches the image, until we update it. So - returning to my original question - is there any way of putting hooks into the slicing such that we can update the metadata for the image when the user does the slicing?

[TomNicholas]

A custom index may allow you to do this in future. You would attach the metadata to the custom index object, so that it is updated after indexing.

see https://github.com/pydata/xarray/projects/1