USAGE.md

Usage

1. Initial Setup

git submodule init
git submodule update
cd vtensorflow/models/research/im2txt
sudo python3 setup.py develop
cd -
cd voicecloning
sudo python3 setup.py develop
cd -
./venv_setup.sh
source ./visual-questioner-env/bin/activate

2. Downloads

• Download Timur's pre-trained im2txt model and the corresponding word_counts.txt.
  • Put all files in the <project root>/im2txt/ directory.
  • Run this code to fix the checkpoint.
• Download the simplified training set from the Natural Questions dataset.
• Download the pre-trained voice cloning models.
• Download male.txt and female.txt and put them in the <project root>/names/ directory.
• Edit config.yaml.

3. Data Generation

Generate GPT-2 training data with

python3 gen_nq_data.py <path_to_natural_questions_jsonl>

4. Training

Train the GPT-2 questioner with train_gpt2_questioner.ipynb.

5. Evaluation

Run the GPT-2 questioner with questioner_gui.py, questioner_cli.py, or run_gpt2_questioner.ipynb.