XAI_EXAMPLE_exMatchina_Text.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "XAI_EXAMPLE_exMatchina_Text.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.9"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/flecue/xai-aaai2021/blob/main/XAI_EXAMPLE_exMatchina_Text.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NkOhduOHKrgT"
      },
      "source": [
        "CREDITS: https://github.com/nesl/ExMatchina/\n",
        "\n",
        "Used as part of AAAI 2021 Tutorial on XAI - https://xaitutorial2021.github.io/\n",
        "                                          - https://github.com/flecue/xai-aaai2021"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DxMAB6BYhu0U"
      },
      "source": [
        "## Mount Drive"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qP-HhgZnhxBb",
        "outputId": "1b842195-08f7-4f32-bc27-b763cddfd588"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/gdrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YjzMTr-Ih66A"
      },
      "source": [
        "## Configuration"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ak-_8K6iiVQ-"
      },
      "source": [
        "1- Make sure to get \"XAI_EXAMPLE_exMatchina_ALL\" from https://github.com/flecue/xai-aaai2021/tree/main/XAI_EXAMPLE_exMatchina_ALL\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zspdc-2NiXs4",
        "outputId": "6d88f94e-9d83-4c7e-cb32-b8db0def2bd9"
      },
      "source": [
        "%cd '/content/gdrive/MyDrive/dev'\n",
        "%ls"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/gdrive/MyDrive/dev\n",
            "ECG-Explanations.ipynb       XAI_EXAMPLE_exMatchina_Image.ipynb\n",
            "\u001b[0m\u001b[01;34mpulse\u001b[0m/                       XAI_EXAMPLE_exMatchina_Text.ipynb\n",
            "\u001b[01;34mXAI_EXAMPLE_exMatchina_ALL\u001b[0m/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "na4eQ5CAicJO",
        "outputId": "17dc823a-bc5f-4fb5-d8a1-74b11721696a"
      },
      "source": [
        "!pip3 install -r XAI_EXAMPLE_exMatchina_ALL/requirements.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting tensorflow==1.15.3\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/02/36/9a02e27f0ec248b676a380ffe910c1858e3af3027c0d4d513dd0b56a5613/tensorflow-1.15.3-cp36-cp36m-manylinux2010_x86_64.whl (110.5MB)\n",
            "\u001b[K     |████████████████████████████████| 110.5MB 95kB/s \n",
            "\u001b[?25hCollecting numpy==1.17.4\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/d2/ab/43e678759326f728de861edbef34b8e2ad1b1490505f20e0d1f0716c3bf4/numpy-1.17.4-cp36-cp36m-manylinux1_x86_64.whl (20.0MB)\n",
            "\u001b[K     |████████████████████████████████| 20.0MB 1.3MB/s \n",
            "\u001b[?25hCollecting gast==0.2.2\n",
            "  Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (0.2.0)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (0.8.1)\n",
            "Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (0.36.2)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.1.2)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.12.4)\n",
            "Collecting tensorboard<1.16.0,>=1.15.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n",
            "\u001b[K     |████████████████████████████████| 3.8MB 46.4MB/s \n",
            "\u001b[?25hRequirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.32.0)\n",
            "Collecting keras-applications>=1.0.8\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 7.8MB/s \n",
            "\u001b[?25hCollecting tensorflow-estimator==1.15.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n",
            "\u001b[K     |████████████████████████████████| 512kB 47.4MB/s \n",
            "\u001b[?25hRequirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.3.0)\n",
            "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.15.0)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.12.1)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.1.0)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (0.10.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (51.1.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.3.3)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (1.0.1)\n",
            "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (2.10.0)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.3.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.4.0)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.3->-r XAI_EXAMPLE_exMatchina_ALL/requirements.txt (line 1)) (3.7.4.3)\n",
            "Building wheels for collected packages: gast\n",
            "  Building wheel for gast (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for gast: filename=gast-0.2.2-cp36-none-any.whl size=7540 sha256=f4331aea3d9e19bf68428f4fa3b108c2234a5ebf1bfda9b8b9d4dc12a39890ca\n",
            "  Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd\n",
            "Successfully built gast\n",
            "\u001b[31mERROR: tensorflow-probability 0.12.1 has requirement gast>=0.3.2, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.\u001b[0m\n",
            "Installing collected packages: gast, numpy, tensorboard, keras-applications, tensorflow-estimator, tensorflow\n",
            "  Found existing installation: gast 0.3.3\n",
            "    Uninstalling gast-0.3.3:\n",
            "      Successfully uninstalled gast-0.3.3\n",
            "  Found existing installation: numpy 1.19.5\n",
            "    Uninstalling numpy-1.19.5:\n",
            "      Successfully uninstalled numpy-1.19.5\n",
            "  Found existing installation: tensorboard 2.4.0\n",
            "    Uninstalling tensorboard-2.4.0:\n",
            "      Successfully uninstalled tensorboard-2.4.0\n",
            "  Found existing installation: tensorflow-estimator 2.4.0\n",
            "    Uninstalling tensorflow-estimator-2.4.0:\n",
            "      Successfully uninstalled tensorflow-estimator-2.4.0\n",
            "  Found existing installation: tensorflow 2.4.0\n",
            "    Uninstalling tensorflow-2.4.0:\n",
            "      Successfully uninstalled tensorflow-2.4.0\n",
            "Successfully installed gast-0.2.2 keras-applications-1.0.8 numpy-1.17.4 tensorboard-1.15.0 tensorflow-1.15.3 tensorflow-estimator-1.15.1\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "numpy"
                ]
              }
            }
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ihYu1QRmie6V",
        "outputId": "cd37324d-7f34-425a-fdc0-b121120de531"
      },
      "source": [
        "%ls XAI_EXAMPLE_exMatchina_ALL/Examples/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[0m\u001b[01;34mdata\u001b[0m/  \u001b[01;34mTrained_Models\u001b[0m/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bN_FPWRXikGS"
      },
      "source": [
        "2 - Make sure to get \"data/\" from https://drive.google.com/drive/folders/1ZRWIeUHxGbKpqWkJ2HpiSLtmUyllfThf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OcCy2ACiLL8f"
      },
      "source": [
        "# Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dsD_Gufchdgk"
      },
      "source": [
        "from warnings import simplefilter \n",
        "simplefilter(action='ignore', category=FutureWarning)\n",
        "\n",
        "import numpy as np\n",
        "import pickle\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "from tensorflow.keras.models import Model, load_model\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "import sys\n",
        "sys.path.append('./XAI_EXAMPLE_exMatchina_ALL')\n",
        "from exmatchina import *"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5l_UpWOHw01J"
      },
      "source": [
        "num_classes = 2\n",
        "class_names = ['negative', 'positive']\n",
        "\n",
        "class_dict = {\n",
        "    'negative': 0,\n",
        "    'positive': 1,\n",
        "}\n",
        "\n",
        "inv_class_dict = {v: k for k, v in class_dict.items()}\n",
        "\n",
        "## These are the randomly generated indices \n",
        "\n",
        "all_idx = np.array([  9528,  11977,  17734,  18431,  19988])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Eq_UjQBgxKAP",
        "outputId": "d6e8382d-021b-43e3-f2cb-b654780b3e7d"
      },
      "source": [
        "raw_images_train = pd.read_pickle('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/X_train_df')\n",
        "raw_images_test = pd.read_pickle('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/X_test_df')\n",
        "\n",
        "with open('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/tokenizer.pickle', 'rb') as f:\n",
        "    tk = pickle.load(f)\n",
        "\n",
        "word_index = tk.word_index\n",
        "print('[INFO] Number of unique tokens found (in train data):', len(word_index))\n",
        "\n",
        "id_to_word = {value:key for key,value in word_index.items()}"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[INFO] Number of unique tokens found (in train data): 283625\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZLfGlYMU99G1"
      },
      "source": [
        "VOCAB_SIZE = 20000\n",
        "MAX_SEQ_LEN = 40\n",
        "EMB_DIM = 100\n",
        "\n",
        "# Returns the raw text of the review\n",
        "def get_review(x):\n",
        "    return ' '.join(id_to_word[id] for id in x if id != 0)\n",
        "\n",
        "def get_train_review(idx):\n",
        "    return (raw_images_train['Texts'][idx]).replace(\"&amp;\",\"&\").replace('&quot;','\"').replace('&lt;','<').replace('_','@')\n",
        "\n",
        "def get_test_review(idx):\n",
        "    return raw_images_test['Texts'][idx].replace(\"&amp;\",\"&\").replace('&quot;','\"').replace('&lt;','<').replace('_','@')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T1iWGXGLy7Q9",
        "outputId": "43222007-3e01-45d6-9735-64b2f2c4937e"
      },
      "source": [
        "X_train = np.load('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/X_train.npy')\n",
        "X_test = np.load('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/X_test.npy')\n",
        "\n",
        "y_train = np.load('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/y_train.npy')\n",
        "y_test = np.load('XAI_EXAMPLE_exMatchina_ALL/Examples/data/text/y_test.npy')\n",
        "\n",
        "print(X_train.shape)\n",
        "print(X_test.shape)\n",
        "print(y_train.shape)\n",
        "print(y_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(1280000, 40)\n",
            "(320000, 40)\n",
            "(1280000, 1)\n",
            "(320000, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-h88vhkc2r7k",
        "outputId": "32e4c8fa-1061-4330-e05c-5ca2c21b0b94"
      },
      "source": [
        "model = load_model('XAI_EXAMPLE_exMatchina_ALL/Examples/Trained_Models/text.hdf5')\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "If using Keras pass *_constraint arguments to layers.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "Embedding (Embedding)        (None, 40, 100)           2000000   \n",
            "_________________________________________________________________\n",
            "Drop_1 (Dropout)             (None, 40, 100)           0         \n",
            "_________________________________________________________________\n",
            "Conv_1 (Conv1D)              (None, 40, 512)           154112    \n",
            "_________________________________________________________________\n",
            "Max_1 (MaxPooling1D)         (None, 20, 512)           0         \n",
            "_________________________________________________________________\n",
            "Drop_2 (Dropout)             (None, 20, 512)           0         \n",
            "_________________________________________________________________\n",
            "Conv_2 (Conv1D)              (None, 20, 256)           393472    \n",
            "_________________________________________________________________\n",
            "Drop_3 (Dropout)             (None, 20, 256)           0         \n",
            "_________________________________________________________________\n",
            "Conv_3 (Conv1D)              (None, 20, 15)            11535     \n",
            "_________________________________________________________________\n",
            "Flatten_1 (Flatten)          (None, 300)               0         \n",
            "_________________________________________________________________\n",
            "Dense_2 (Dense)              (None, 20)                6020      \n",
            "_________________________________________________________________\n",
            "Output (Dense)               (None, 1)                 21        \n",
            "=================================================================\n",
            "Total params: 2,565,160\n",
            "Trainable params: 2,565,160\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "710HteKChdgy"
      },
      "source": [
        "# ExMatchina"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ozHCy15shdgz",
        "outputId": "db700929-62d7-4e28-e2dd-7174c4547372"
      },
      "source": [
        "selected_layer = 'Flatten_1'\n",
        "\n",
        "exm = ExMatchina(model=model, layer=selected_layer, examples=X_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Getting activations...\n",
            "Getting labels...\n",
            "Generating activation matrix...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VlRSDxqghdg6",
        "outputId": "a2f034c0-ef4f-4cec-e59c-b835f3b86459"
      },
      "source": [
        "for test_idx in all_idx:\n",
        "    test_input = X_test[test_idx]\n",
        "    \n",
        "    to_explain = np.expand_dims(test_input, axis=0)\n",
        "    class_pred = np.rint(model.predict(to_explain)[0])\n",
        "    print(inv_class_dict[class_pred[0]])\n",
        "    \n",
        "    (examples, indices) = exm.return_nearest_examples(test_input, 3)\n",
        "    \n",
        "    review = get_test_review(test_idx)\n",
        "    # print(\"REVIEW RAW\", review)\n",
        "    review = get_test_review(test_idx)\n",
        "    review_1 = get_train_review(indices[0])\n",
        "    review_2 = get_train_review(indices[1])\n",
        "    review_3 = get_train_review(indices[2])\n",
        "\n",
        "    print(test_idx, \"REVIEW:\", review)\n",
        "    print(test_idx, \"Example 1:\", review_1)\n",
        "    print(test_idx, \"Example 2:\", review_2)\n",
        "    print(test_idx, \"Example 3:\", review_3)\n",
        "    print(\"\\n=====\\n\")\n",
        "#     draw_txt(review, \"text-\" + str(test_idx))\n",
        "#     draw_txt( \"SIMILAR TWEET #1 (\" + sentiment + \"):\\n\" +\n",
        "#              review_1 + \"\\n\\nSIMILAR TWEET #2 (\" + sentiment + \"):\\n\" +\n",
        "#              review_2 + \"\\n\\nSIMILAR TWEET #3 (\" + sentiment + \"):\\n\" + \n",
        "#              review_3 , \"text-\"+ str(test_idx) +\"-example\")\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "negative\n",
            "9528 REVIEW: I dont know.. they said it will arrived 1-2 days,, in fact malah suka lewat, even 4 days,,\n",
            "9528 Example 1: Just missing each other by a week or so.  Have fun while you're in Cancun!\n",
            "9528 Example 2: Oh no  Hope everything goes Ok. Will have lots of positive thoughts for you guys.\n",
            "9528 Example 3: its not fair  mourning for south east asian blockheads..thx 4 share it with us gal\n",
            "\n",
            "=====\n",
            "\n",
            "negative\n",
            "11977 REVIEW: Those of you that know me, say a prayer for my Dad, his heart is broken...\n",
            "11977 Example 1: so i went today to pick up that lap top... my friend talked me out of it  i came home dissapointed\n",
            "11977 Example 2: Yeah its totally crazy. I have friends that are firefighters for Grand Blanc and Mundy. I hear lots of stuff. Real sad.\n",
            "11977 Example 3: Ok, so I got up on my one day off and my sister is apparently sick....  .....\n",
            "\n",
            "=====\n",
            "\n",
            "positive\n",
            "17734 REVIEW: I love being accused of using an aimbot, it's just so flattering.\n",
            "17734 Example 1: well good feedback means so much more,in my opinion. Congratulations\n",
            "17734 Example 2: its fine  i will tomorrow shall be a better day, thankyou.\n",
            "17734 Example 3: Thats ironic cuz I was just watching that. It's really good\n",
            "\n",
            "=====\n",
            "\n",
            "negative\n",
            "18431 REVIEW: you keep disappearing and it makes me a sad panda\n",
            "18431 Example 1: the end of him and me. very sad ending.\n",
            "18431 Example 2: Of to work, going to be a very sad day\n",
            "18431 Example 3: yeah so its been half an hour and still no reply\n",
            "\n",
            "=====\n",
            "\n",
            "positive\n",
            "19988 REVIEW: The road to success is dotted with so many parking places. Agree? Good Tuesday morning Twitter Universe!\n",
            "19988 Example 1: Barack Obama's speech rocked! I've never seen him smile so much. He loves ASU.\n",
            "19988 Example 2: Its so smooth i just wanna pet it - allison. Haha gotta love science class\n",
            "19988 Example 3: She sounds strong-willed and determined.  Can't she just become an engineer?  Without the navy\n",
            "\n",
            "=====\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JW05Rwe0hdg-"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dl6u0t-UhdhB"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sr1eCyLZhdhD"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}