biggan_music_visualizer.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# BigGAN Music Visualizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "In order to utilize the computing resources of Colab GPUs, you'll need to enable GPUs for this notebook:\n",
        "- Navigate to Edit→Notebook Settings\n",
        "- select GPU from the Hardware Accelerator drop-down"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To run the visualizer, first upload an audio file (mp3, wav, etc.) using the `Files` tab in the sidebar."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Install Requirements"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BMieVKFeviC1"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/rushk014/biggan-visualizer.git\n",
        "%cd biggan-visualizer/\n",
        "!pip install -r requirements.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run Visualizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "See the [README](https://github.com/rushk014/biggan-visualizer/blob/master/README.md) for description of input parameters. The only required parameter is `--song SONG`. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "For an uploaded file `example.mp3` (in the default `content/` folder), running the script with default parameters is given in the cell below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bhHmVeQ8vzC9"
      },
      "outputs": [],
      "source": [
        "!python visualizer.py --song ../example.mp3 --batch_size 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Note: If you see the following error, try using a smaller batch size with `--batch_size BATCH_SIZE` (default is `20`) \n",
        "```python\n",
        "RuntimeError: CUDA out of memory. Tried to allocate _ MiB\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Note: Processing take a while, but can be sped up at the cost of resolution or fps by adjusting `--resolution RESOLUTION` (default is $512$, options are $\\{128, 256, 512\\}$) or increasing `--frame_length` (default is 512, range is multiples of $2^6$)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Choose Image Classes using Lyrics\n",
        "\n",
        "To choose the image classes using similarity to the song's lyrics, upload a text file containing the lyrics (eg. `example.txt`) and run the following cell:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!python visualizer.py --song ../example.mp3 --lyrics ../example.txt --batch_size 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Download Visualization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Run the following cell to download the BigGAN visualization output `example.mp4` for input file `example.mp3`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from google.colab import files\n",
        "files.download('output/example.mp4')"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "biggan-music-visualizer.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}