C4_W2_Lab_4_FashionMNIST_CNNAutoEncoder.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "C4_W2_Lab_4_FashionMNIST_CNNAutoEncoder.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Svyum7G3gHcK"
      },
      "source": [
        "## Ungraded Lab: Convolutional Autoencoders\n",
        "\n",
        "In this lab, you will use convolution layers to build your autoencoder. This usually leads to better results than dense networks and you will see it in action with the [Fashion MNIST dataset](https://www.tensorflow.org/datasets/catalog/fashion_mnist)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jk0Tld-U5XFD"
      },
      "source": [
        "## Imports"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3EXwoz-KHtWO"
      },
      "source": [
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass\n",
        "  \n",
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0WGuXlw5bK-"
      },
      "source": [
        "## Prepare the Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aTySDKEhLNLY"
      },
      "source": [
        "As before, you will load the train and test sets from TFDS. Notice that we don't flatten the image this time. That's because we will be using convolutional layers later that can deal with 2D images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t9F7YsCNIKSA"
      },
      "source": [
        "def map_image(image, label):\n",
        "  '''Normalizes the image. Returns image as input and label.'''\n",
        "  image = tf.cast(image, dtype=tf.float32)\n",
        "  image = image / 255.0\n",
        "\n",
        "  return image, image"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9ZsciqJXL368"
      },
      "source": [
        "BATCH_SIZE = 128\n",
        "SHUFFLE_BUFFER_SIZE = 1024\n",
        "\n",
        "train_dataset = tfds.load('fashion_mnist', as_supervised=True, split=\"train\")\n",
        "train_dataset = train_dataset.map(map_image)\n",
        "train_dataset = train_dataset.shuffle(SHUFFLE_BUFFER_SIZE).batch(BATCH_SIZE).repeat()\n",
        "\n",
        "test_dataset = tfds.load('fashion_mnist', as_supervised=True, split=\"test\")\n",
        "test_dataset = test_dataset.map(map_image)\n",
        "test_dataset = test_dataset.batch(BATCH_SIZE).repeat()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uoyz09uKMDn5"
      },
      "source": [
        "## Define the Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V1-Fw_qnZPV7"
      },
      "source": [
        "As mentioned, you will use convolutional layers to build the model. This is composed of three main parts: encoder, bottleneck, and decoder. You will follow the configuration shown in the image below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "568W0TYyY9nl"
      },
      "source": [
        "<img src=\"https://drive.google.com/uc?export=view&id=15zh7bst9KKvciRdCvMAH7kXt3nNkABzO\" width=\"75%\" height=\"75%\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O2IvtyIoZnb4"
      },
      "source": [
        "The encoder, just like in previous labs, will contract with each additional layer. The features are generated with the Conv2D layers while the max pooling layers reduce the dimensionality."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wxh8h-UMk2iL"
      },
      "source": [
        "def encoder(inputs):\n",
        "  '''Defines the encoder with two Conv2D and max pooling layers.'''\n",
        "  conv_1 = tf.keras.layers.Conv2D(filters=64, kernel_size=(3,3), activation='relu', padding='same')(inputs)\n",
        "  max_pool_1 = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(conv_1)\n",
        "\n",
        "  conv_2 = tf.keras.layers.Conv2D(filters=128, kernel_size=(3,3), activation='relu', padding='same')(max_pool_1)\n",
        "  max_pool_2 = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(conv_2)\n",
        "\n",
        "  return max_pool_2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g9KQYnabazLl"
      },
      "source": [
        "A bottleneck layer is used to get more features but without further reducing the dimension afterwards. Another layer is inserted here for visualizing the encoder output."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wRWmLA3VliDr"
      },
      "source": [
        "def bottle_neck(inputs):\n",
        "  '''Defines the bottleneck.'''\n",
        "  bottle_neck = tf.keras.layers.Conv2D(filters=256, kernel_size=(3,3), activation='relu', padding='same')(inputs)\n",
        "  encoder_visualization = tf.keras.layers.Conv2D(filters=1, kernel_size=(3,3), activation='sigmoid', padding='same')(bottle_neck)\n",
        "\n",
        "  return bottle_neck, encoder_visualization"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FayvcE3ebZxk"
      },
      "source": [
        "The decoder will upsample the bottleneck output back to the original image size."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XZgLt5uAmArk"
      },
      "source": [
        "def decoder(inputs):\n",
        "  '''Defines the decoder path to upsample back to the original image size.'''\n",
        "  conv_1 = tf.keras.layers.Conv2D(filters=128, kernel_size=(3,3), activation='relu', padding='same')(inputs)\n",
        "  up_sample_1 = tf.keras.layers.UpSampling2D(size=(2,2))(conv_1)\n",
        "\n",
        "  conv_2 = tf.keras.layers.Conv2D(filters=64, kernel_size=(3,3), activation='relu', padding='same')(up_sample_1)\n",
        "  up_sample_2 = tf.keras.layers.UpSampling2D(size=(2,2))(conv_2)\n",
        "\n",
        "  conv_3 = tf.keras.layers.Conv2D(filters=1, kernel_size=(3,3), activation='sigmoid', padding='same')(up_sample_2)\n",
        "\n",
        "  return conv_3"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dvfhvk9qbvCp"
      },
      "source": [
        "You can now build the full autoencoder using the functions above."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fQKwO64iiOYl"
      },
      "source": [
        "def convolutional_auto_encoder():\n",
        "  '''Builds the entire autoencoder model.'''\n",
        "  inputs = tf.keras.layers.Input(shape=(28, 28, 1,))\n",
        "  encoder_output = encoder(inputs)\n",
        "  bottleneck_output, encoder_visualization = bottle_neck(encoder_output)\n",
        "  decoder_output = decoder(bottleneck_output)\n",
        "  \n",
        "  model = tf.keras.Model(inputs =inputs, outputs=decoder_output)\n",
        "  encoder_model = tf.keras.Model(inputs=inputs, outputs=encoder_visualization)\n",
        "  return model, encoder_model\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1MmS7r0tkuIf"
      },
      "source": [
        "convolutional_model, convolutional_encoder_model = convolutional_auto_encoder()\n",
        "convolutional_model.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "convolutional_encoder_model.summary()"
      ],
      "metadata": {
        "id": "pILQPcZ-kTfA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5FRxRr0LMLCs"
      },
      "source": [
        "## Compile and Train the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J0Umj_xaiHL_"
      },
      "source": [
        "train_steps = 60000 // BATCH_SIZE\n",
        "valid_steps = 60000 // BATCH_SIZE\n",
        "\n",
        "convolutional_model.compile(optimizer=tf.keras.optimizers.Adam(), \n",
        "                            loss='binary_crossentropy')\n",
        "conv_model_history = convolutional_model.fit(train_dataset,\n",
        "                                             steps_per_epoch=train_steps, \n",
        "                                             validation_data=test_dataset,\n",
        "                                             validation_steps=valid_steps,\n",
        "                                             epochs=40)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-8zE9OiAMUd7"
      },
      "source": [
        "## Display sample results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DCUOM7F_cf26"
      },
      "source": [
        "As usual, let's see some sample results from the trained model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A35RlIqKIsQv"
      },
      "source": [
        "def display_one_row(disp_images, offset, shape=(28, 28)):\n",
        "  '''Display sample outputs in one row.'''\n",
        "  for idx, test_image in enumerate(disp_images):\n",
        "    plt.subplot(3, 10, offset + idx + 1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    test_image = np.reshape(test_image, shape)\n",
        "    plt.imshow(test_image, cmap='gray')\n",
        "\n",
        "\n",
        "def display_results(disp_input_images, disp_encoded, disp_predicted, enc_shape=(8,4)):\n",
        "  '''Displays the input, encoded, and decoded output values.'''\n",
        "  plt.figure(figsize=(15, 5))\n",
        "  display_one_row(disp_input_images, 0, shape=(28,28,))\n",
        "  display_one_row(disp_encoded, 10, shape=enc_shape)\n",
        "  display_one_row(disp_predicted, 20, shape=(28,28,))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qtQyQRxRN_hH"
      },
      "source": [
        "# take 1 batch of the dataset\n",
        "test_dataset = test_dataset.take(1)\n",
        "\n",
        "# take the input images and put them in a list\n",
        "output_samples = []\n",
        "for input_image, image in tfds.as_numpy(test_dataset):\n",
        "      output_samples = input_image\n",
        "\n",
        "# pick 10 indices\n",
        "idxs = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])\n",
        "\n",
        "# prepare test samples as a batch of 10 images\n",
        "conv_output_samples = np.array(output_samples[idxs])\n",
        "conv_output_samples = np.reshape(conv_output_samples, (10, 28, 28, 1))\n",
        "\n",
        "# get the encoder ouput\n",
        "encoded = convolutional_encoder_model.predict(conv_output_samples)\n",
        "\n",
        "# get a prediction for some values in the dataset\n",
        "predicted = convolutional_model.predict(conv_output_samples)\n",
        "\n",
        "# display the samples, encodings and decoded values!\n",
        "display_results(conv_output_samples, encoded, predicted, enc_shape=(7,7))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "5r3JqLjykdBz"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}