Using calibration video for multiview PCA loss

[olivier-cuttlefish]

Given that one already have calibration video with checkerboards, one could technically use those points to compute the projection matrix from one view to the other one. I would assume that it would give better results than using labeling data that can be less precise and less good at extrapolating throughout the view (for freely moving animal setups). Do you expect it to substantially improve performance or only anecdotally ? What would be the recommended way to bypass using labeled data and force the lightningpose to use checkerboard data instead ?

[themattinthehatt]

That's a great suggestion - I do expect that the checkboard data would be very helpful for computing the multiview PCA subspace, especially in the low-label regime.

In the new year we are going to start focusing more of our time on building out features for the multiview case, so this is certainly something we could work on.

If you don't mind me asking, what lab are you from, and what does your setup look like (species, number of cameras, etc.)? As we move forward with providing more support for multiview setups it will be helpful for us to have an idea of who is using lightning pose and on what data.

[olivier-cuttlefish]

Great to hear that you are planning to add more features! Would that be OK for me to answer you directly by email ?

[themattinthehatt]

yes definitely - m.whiteway at columbia dot edu