Want to train VSR model for digit recognition using grid dataset.

[Chhaya-Trootech]

Thank you for sharing the code.

I am interested in training VSR model (lip_reading) for digit recognition using grid dataset. because the pre-train weights are not perfectly working for digits recognition. this repo gives idea for training lrs2 nd lrs3. how can i use this to train model for grid dataset?

[mpc001]

Hi @Chhaya-Trootech, I recommend building your own vocabulary list (either subword-based or charater-based) using GRID. An example instruction can be found at https://github.com/mpc001/auto_avsr/blob/main/INSTRUCTION.md. Then, fine-tune using the customized vocabulary list on GRID with the provided pre-trained checkpoint. It is also important to replace with your customized dictionary at the inference stage. https://github.com/mpc001/auto_avsr/blob/main/datamodule/transforms.py#L20-L32