submission5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kaggle Titanic Prediction Competition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intro\n",
    "\n",
    "### Competition Description\n",
    "The sinking of the RMS Titanic is one of the most infamous shipwrecks in history. On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This sensational tragedy shocked the international community and led to better safety regulations for ships.\n",
    "\n",
    "One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.\n",
    "\n",
    "In this challenge, we ask you to complete the analysis of what sorts of people were likely to survive. In particular, we ask you to apply the tools of machine learning to predict which passengers survived the tragedy.\n",
    "\n",
    "Webpage: https://www.kaggle.com/c/titanic/data?select=test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data and libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import random as rnd\n",
    "\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.linear_model import Perceptron\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (891, 12)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load train data\n",
    "train_df = pd.read_csv('train.csv')\n",
    "print('Shape: ', train_df.shape)\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape:  (418, 11)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>Wirz, Mr. Albert</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315154</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Mrs. Alexander (Helga E Lindqvist)</td>\n",
       "      <td>female</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                          Name     Sex  \\\n",
       "0          892       3                              Kelly, Mr. James    male   \n",
       "1          893       3              Wilkes, Mrs. James (Ellen Needs)  female   \n",
       "2          894       2                     Myles, Mr. Thomas Francis    male   \n",
       "3          895       3                              Wirz, Mr. Albert    male   \n",
       "4          896       3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)  female   \n",
       "\n",
       "    Age  SibSp  Parch   Ticket     Fare Cabin Embarked  \n",
       "0  34.5      0      0   330911   7.8292   NaN        Q  \n",
       "1  47.0      1      0   363272   7.0000   NaN        S  \n",
       "2  62.0      0      0   240276   9.6875   NaN        Q  \n",
       "3  27.0      0      0   315154   8.6625   NaN        S  \n",
       "4  22.0      1      1  3101298  12.2875   NaN        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load test data\n",
    "test_df = pd.read_csv('test.csv')\n",
    "print('Shape: ', test_df.shape)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  891 non-null    int64  \n",
      " 1   Survived     891 non-null    int64  \n",
      " 2   Pclass       891 non-null    int64  \n",
      " 3   Name         891 non-null    object \n",
      " 4   Sex          891 non-null    object \n",
      " 5   Age          714 non-null    float64\n",
      " 6   SibSp        891 non-null    int64  \n",
      " 7   Parch        891 non-null    int64  \n",
      " 8   Ticket       891 non-null    object \n",
      " 9   Fare         891 non-null    float64\n",
      " 10  Cabin        204 non-null    object \n",
      " 11  Embarked     889 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n",
      "------------------------------\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 418 entries, 0 to 417\n",
      "Data columns (total 11 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  418 non-null    int64  \n",
      " 1   Pclass       418 non-null    int64  \n",
      " 2   Name         418 non-null    object \n",
      " 3   Sex          418 non-null    object \n",
      " 4   Age          332 non-null    float64\n",
      " 5   SibSp        418 non-null    int64  \n",
      " 6   Parch        418 non-null    int64  \n",
      " 7   Ticket       418 non-null    object \n",
      " 8   Fare         417 non-null    float64\n",
      " 9   Cabin        91 non-null     object \n",
      " 10  Embarked     418 non-null    object \n",
      "dtypes: float64(2), int64(4), object(5)\n",
      "memory usage: 36.0+ KB\n"
     ]
    }
   ],
   "source": [
    "# first view of dataset\n",
    "train_df.info()\n",
    "print('-'*30)\n",
    "test_df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.describe() \n",
    "# test_df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Passenger_Id\n",
    "Observation: PassengerId doesn't seem to have significant correlation with survival"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7ff0c3b36c70>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd4AAADQCAYAAACp4wAHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAATsElEQVR4nO3da7BlZZ3f8e9PLiMXoxAPVAucgIi3kLHBEwTJjETAtJcZNAYHRx2cENtUyQQnTE2hk6pp8mZIBSFWRilbJXQcRmgQY5czokyPRLCwudlc2kaxgMHGHpo2qDAxON3882KtHo7t6T6rz95nnbN3fz9Vp/Zez16XZz+c0z/WWs96nlQVkiSpH89b6ApIkrQ3MXglSeqRwStJUo8MXkmSemTwSpLUo337PNiyZcvqxhtv7POQkqTRkYWuQB96PePdunVrn4eTJGnR8VKzJEk9MnglSeqRwStJUo9mDd4kz09ye5J7kmxIcnFbviLJY0nWtz9vmf/qSpI02rr0an4GeGNVPZ1kP+DWJF9pP7u8qi6dv+pJkjReZg3eamZReLpd3K/9cWYFSZLmoNNzvEn2Ae4CXgZ8oqrWJXkzcH6S3wHuBC6sqidn2HY5sBxgcnJyaBVfTC6/6XsD7+P3z3z5EGoiSVrsOnWuqqrtVbUUOBI4KcnxwBXAscBSYDPwsV1su7KqpqpqamJiYiiVliRpVO1Rr+aq+jFwM7Csqh5vA/lZ4NPAScOvniRJ46VLr+aJJC9q3x8AnAE8kGTJtNXeAdw/LzWUJGmMdLnHuwRY1d7nfR6wuqq+nORzSZbSdLR6BPjgvNVSkqQx0aVX873ACTOUv29earSXGkYHLbCTliQtdo5cJUlSjwxeSZJ6ZPBKktQjg1eSpB4ZvJIk9cjglSSpRwavJEk9MnglSeqRwStJUo8MXkmSemTwSpLUI4NXkqQedZmdSJJ6N44Th4zjd9Ke6zIf7/OT3J7kniQbklzclh+a5KYkD7avh8x/dSVJGm1dLjU/A7yxql4DLAWWJTkZuAhYW1XHAWvbZUmStBuzBm81nm4X92t/CjgLWNWWrwLePh8VlCRpnHS6x5tkH+Au4GXAJ6pqXZLDq2ozQFVtTnLYLrZdDiwHmJycHE6tNe+8FyVJ86NTr+aq2l5VS4EjgZOSHN/1AFW1sqqmqmpqYmJijtWUJGk87NHjRFX1Y+BmYBnweJIlAO3rlmFXTpKkcdOlV/NEkhe17w8AzgAeANYA57arnQt8aZ7qKEnS2Ohyj3cJsKq9z/s8YHVVfTnJbcDqJOcBjwJnz2M9JUkaC7MGb1XdC5wwQ/mPgNPno1KSurMjnDRaHDJSkqQeGbySJPXI4JUkqUdOkiDNwTDuq3pPdbQM6176MCymuoC/y3vKM15Jknpk8EqS1CODV5KkHhm8kiT1yM5V0gJZbB1kJPXDM15Jknpk8EqS1CODV5KkHu3V93jH8R7bOH4njZbF9ju42OojdZmP96gkX0+yMcmGJBe05SuSPJZkffvzlvmvriRJo63LGe824MKqujvJC4C7ktzUfnZ5VV06f9WTJGm8dJmPdzOwuX3/VJKNwBHzXTFJksbRHnWuSnI0cAKwri06P8m9Sa5McsiwKydJ0rjpHLxJDga+AHy4qn4KXAEcCyylOSP+2C62W57kziR3PvHEE4PXWJKkEdYpeJPsRxO6V1fVDQBV9XhVba+qZ4FPAyfNtG1VrayqqaqampiYGFa9JUkaSV16NQf4LLCxqi6bVr5k2mrvAO4ffvUkSRovXXo1nwq8D7gvyfq27KPAu5MsBQp4BPjgPNRPkqSx0qVX861AZvjoL4dfnW58IH6EfP1PhrKby7e9cyj70a75dyX1wyEjJUnqkcErSVKPDF5Jknpk8EqS1CODV5KkHhm8kiT1yOCVJKlHBq8kST0yeCVJ6pHBK0lSjwxeSZJ6ZPBKktSjLrMTLTonP7pyKPv51uTyoexH88//5vPPNpb64RmvJEk9mjV4kxyV5OtJNibZkOSCtvzQJDclebB9PWT+qytJ0mjrcsa7Dbiwql4FnAx8KMmrgYuAtVV1HLC2XZYkSbsxa/BW1eaqurt9/xSwETgCOAtY1a62Cnj7PNVRkqSxsUedq5IcDZwArAMOr6rN0IRzksN2sc1yYDnA5OTkQJXV6LntoR8tdBV+gR2IJC20zp2rkhwMfAH4cFX9tOt2VbWyqqaqampiYmIudZQkaWx0Ct4k+9GE7tVVdUNb/HiSJe3nS4At81NFSZLGR5dezQE+C2ysqsumfbQGOLd9fy7wpeFXT5KkRpI/ap+uuTfJ+iSvG8I+fzPJUDoHJ3m6y3pd7vGeCrwPuC/J+rbso8AlwOok5wGPAmfPoZ4aMu9hShpHSU4B3gacWFXPJHkxsH/Hbfetqm0zfVZVa2hOJHsza/BW1a1AdvHx6cOtjiRJM1oCbK2qZwCqaitAkkeAqarammQKuLSqTkuyAngJcDSwNcmxwL+tqg3tdjcDFwL/DJgC/gi4B3hpVT2b5EDgu8BLgUngE8AE8H+BD1TVA0mOAf6cJktv7PpFHLlKkjQKvgYcleR7ST6Z5A0dtnktcFZV/TZwDfAu+Id+SS+pqrt2rFhVP6EJ3h37/Q3gq1X198BK4Peq6rXAHwCfbNf5OHBFVf1z4G+7fhGDV5K06FXV0zRBuhx4Arg2yftn2WxNVf2sfb+a526Jvgu4bob1rwV+q31/TnuMg4HXA9e1t1s/RXP2Dc2t2M+37z/X9buM5CQJkqS9T1VtB24Gbk5yH03H3m08dxL5/J02+btp2z6W5EdJfpUmXD84wyHWAH+S5FCakP9r4CDgx1W1dFfV2tPvsVcHrx2RtJDG9fdvsX2vYdRnsbXx3ijJK4Bnq+rBtmgp8DfAATQh+RXgnbPs5hrgD4EXVtV9O39YVU8nuZ3mEvKX26D/aZKHk5xdVde1T/r8alXdA3yT5sz4z4D3dP0uXmqWJI2Cg4FVSb6T5F7g1cAK4GLg40luAbbPso/raYJy9W7WuRZ4b/u6w3uA85LcA2ygGTIZ4AKa+QvuAF7Y9Yvs1We8kqTR0HaEev0MH90CvHyG9VfMUPY4O+VeVV0FXDVt+Xp2epKnqh4Gls2wv4eBU6YVXbLrb/Acz3glSeqRZ7ySNItxvG89XJcudAVGime8kiT1yOCVJKlHBq8kST0yeCVJ6pGdq6Q5WHydWyTNtyTLaAbX2Af4TFV1enxoZ7MGb5IraaZi2lJVx7dlK4AP0IyXCfDRqvrLuVRAkqQ9dfRFf7FimPt75JK37nZ/SfahmaHoTGATcEeSNVX1nT09VpdLzVcxw4PDwOVVtbT9MXQlSePsJOD7VfVQVf2cZvjJs2bZZkazBm9VfQP4P3PZuSRJY+II4AfTlje1ZXtskHu85yf5HeBO4MKqenKmlZIsp5nGicnJyQEOpz55D3N0+N9K6kVmKNvjmYlg7r2arwCOpZkdYjPwsV2tWFUrq2qqqqYmJibmeDhJkhbUJuCoactHAj+cy47mFLxV9XhVba+qZ4FP01z7liRpXN0BHJfkmCT708xytGYuO5pT8CZZMm3xHcD9c9mPJEmjoKq2AecDXwU2AqurasNc9tXlcaLPA6cBL06yCfhj4LQkS2mubz8CfHAuB5ckaS5me/xnPrRP8Az8FM+swVtV756h+LODHliSdsdOYxpXDhkpSVKPDF5Jknpk8EqS1CMnSZCknnjfWuAZryRJvTJ4JUnqIMmVSbYkGWjsCi81S5JGz4oXrhju/n7SZX9XAX8K/M9BDuUZryRJHQxrtj6DV5KkHhm8kiT1yOCVJKlHBq8kST0yeCVJ6qCdre824BVJNiU5by776TIt4JXA24AtVXV8W3YocC1wNM20gO+qqifnUgFJkvZYt8d/hmoXs/XtsS5nvFcBy3YquwhYW1XHAWvbZUmSNItZg3cXzy2dBaxq368C3j7cakmSNJ7mOnLV4VW1GaCqNic5bFcrJlkOLAeYnJyc4+EWNwc+lyR1Ne+dq6pqZVVNVdXUxMTEfB9OkqRFba7B+3iSJQDt65bhVUmSpPE11+BdA5zbvj8X+NJwqiNJ0nibNXh38dzSJcCZSR4EzmyXJUnSLGbtXLWb55ZOH3JdJEkae45cJUlSjwxeSZJ6ZPBKktQjg1eSpB4ZvJIk9cjglSSpRwavJEk9MnglSeqRwStJUo8MXkmSemTwSpLUI4NXkqQeGbySJPVo1tmJdifJI8BTwHZgW1VNDaNSkiSNq4GCt/Uvq2rrEPYjSdLY81KzJEk9GjR4C/hakruSLB9GhSRJGmeDXmo+tap+mOQw4KYkD1TVN6av0AbycoDJyckBDydJ0mgb6Iy3qn7Yvm4BvgicNMM6K6tqqqqmJiYmBjmcJEkjb87Bm+SgJC/Y8R54E3D/sComSdI4GuRS8+HAF5Ps2M+fV9WNQ6mVJEljas7BW1UPAa8ZYl0kSRp7Pk4kSVKPDF5Jknpk8EqS1CODV5KkHhm8kiT1yOCVJKlHBq8kST0yeCVJ6pHBK0lSjwxeSZJ6ZPBKktQjg1eSpB4ZvJIk9cjglSSpRwMFb5JlSb6b5PtJLhpWpSRJGldzDt4k+wCfAN4MvBp4d5JXD6tikiSNo0HOeE8Cvl9VD1XVz4FrgLOGUy1JksbTvgNsewTwg2nLm4DX7bxSkuXA8nbx6STfHeCYAC8Gtg64j72dbTgY228wtt/gFlcb/ruPDWtPN1bVsmHtbLEaJHgzQ1n9UkHVSmDlAMf5xYMmd1bV1LD2tzeyDQdj+w3G9hucbTjaBrnUvAk4atrykcAPB6uOJEnjbZDgvQM4LskxSfYHzgHWDKdakiSNpzlfaq6qbUnOB74K7ANcWVUbhlazXRvaZeu9mG04GNtvMLbf4GzDEZaqX7otK0mS5okjV0mS1CODV5KkHo1U8DpE5eySHJXk60k2JtmQ5IK2/NAkNyV5sH09ZNo2H2nb9LtJ/tXC1X7xSLJPkm8n+XK7bPt1lORFSa5P8kD7e3iK7bdnkvx++/d7f5LPJ3m+bTg+RiZ4HaKys23AhVX1KuBk4ENtO10ErK2q44C17TLtZ+cA/xRYBnyybeu93QXAxmnLtl93H6cZCOGVwGto2tH26yjJEcB/AKaq6niazqvnYBuOjZEJXhyispOq2lxVd7fvn6L5R+8ImrZa1a62Cnh7+/4s4JqqeqaqHga+T9PWe60kRwJvBT4zrdj26yDJPwJ+HfgsQFX9vKp+jO23p/YFDkiyL3AgzRgJtuGYGKXgnWmIyiMWqC4jIcnRwAnAOuDwqtoMTTgDh7Wr2a6/7L8Bfwg8O63M9uvmpcATwP9oL9V/JslB2H6dVdVjwKXAo8Bm4CdV9TVsw7ExSsHbaYhKNZIcDHwB+HBV/XR3q85Qtte2a5K3AVuq6q6um8xQtte2H82Z2onAFVV1AvB3tJdEd8H220l77/Ys4BjgJcBBSd67u01mKNur23CxG6XgdYjKjpLsRxO6V1fVDW3x40mWtJ8vAba05bbrLzoV+M0kj9Dcznhjkj/D9utqE7Cpqta1y9fTBLHt190ZwMNV9URV/T1wA/B6bMOxMUrB6xCVHSQJzf21jVV12bSP1gDntu/PBb40rfycJL+S5BjgOOD2vuq72FTVR6rqyKo6muZ37K+r6r3Yfp1U1d8CP0jyirbodOA72H574lHg5CQHtn/Pp9P01bANx8QgsxP1agGHqBw1pwLvA+5Lsr4t+yhwCbA6yXk0f9hnA1TVhiSraf5x3AZ8qKq2917rxc/26+73gKvb/0F+CPhdmv/Jt/06qKp1Sa4H7qZpk2/TDBF5MLbhWHDISEmSejRKl5olSRp5Bq8kST0yeCVJ6pHBK0lSjwxeSZJ6ZPBKQJLtSda3s8Fcl+TAha7TXCW5OcnUDOXvT/KnC1EnSc8xeKXGz6pqaTsbzM+Bf7/QFZoLZ6WRFj+DV/pltwAvS/IbSda1g/3/VZLDAZK8oT07Xt9+9oIkS5J8Y9pZ86+1674pyW1J7m7PpA9uyx9JcnFbfl+SV7blE+1cq3cn+VSSv0ny4vaz9ya5vT3Gp3aEbJKnk/znJOuAU6Z/kSS/m+R7Sf43zeAqkhaYwStN007D9mbgPuBW4OR2sP9raGYsAvgDmtGBlgK/BvwM+G3gq23Za4D1bWD+J+CMqjoRuBP4j9MOt7Utv6LdJ8Af0wxTeSLwRWCyrdergN8CTm2PsR14T7vNQcD9VfW6qrp12ndZAlxME7hn0sxjLWmBjcyQkdI8O2DaEJu30Ix3/Qrg2jbA9gcebj//JnBZkquBG6pqU5I7gCvbCSr+V1WtT/IGmrD7ZjPkLvsDt0075o4JLO4C/nX7/l8A7wCoqhuTPNmWnw68Frij3dcBPDdI/naaSTF29jrg5qp6AiDJtcDL96hVJA2dwSs1ftaeSf6DJP8duKyq1iQ5DVgBUFWXJPkL4C3At5KcUVXfSPLrwFuBzyX5r8CTwE1V9e5dHPOZ9nU7z/0tzjTF247yVVX1kRk++3+7GZvXMWGlRcZLzdKuvRB4rH2/Y1YYkhxbVfdV1X+huXz8yiT/hGYe30/TnC2fCHwLODXJy9rtDkwy2xnnrcC72vXfBBzSlq8F/k2Sw9rPDm2PuTvrgNOS/OP2TPzsTt9a0rwyeKVdWwFcl+QWYOu08g+3Hajuobm/+xXgNJr7ut8G3gl8vL3E+37g80nupQniV85yzIuBNyW5m+Ze82bgqar6Ds394q+1+7oJWLK7HVXV5vY73Ab8Fc1sN5IWmLMTSYtIkl8BtrfTYJ4CXLHzJXBJo817vNLiMkkz5+rzaJ4n/sAC10fSkHnGK0lSj7zHK0lSjwxeSZJ6ZPBKktQjg1eSpB4ZvJIk9ej/A82cUFcB5mZuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 483.875x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "grid = sns.FacetGrid(train_df, hue='Survived', aspect=2)\n",
    "grid.map(plt.hist, 'PassengerId', bins=20, alpha = 0.5)\n",
    "grid.add_legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pclass\n",
    "Observation: Survival rate reduces as Pclass increases. Survival rate esp low in Pclass 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.629630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.472826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.242363</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Survived\n",
       "Pclass          \n",
       "1       0.629630\n",
       "2       0.472826\n",
       "3       0.242363"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[['Pclass','Survived']].groupby('Pclass').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sex\n",
    "Observation: Women way more likely to survive"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>female</th>\n",
       "      <td>0.742038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>male</th>\n",
       "      <td>0.188908</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Survived\n",
       "Sex             \n",
       "female  0.742038\n",
       "male    0.188908"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[['Sex','Survived']].groupby('Sex').mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create Sex_code column in train and test set\n",
    "train_df['Sex_bin'] = train_df['Sex'].map( {'female': 1, 'male': 0} ).astype(int)\n",
    "test_df['Sex_bin'] = test_df['Sex'].map( {'female': 1, 'male': 0} ).astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Age\n",
    "Observation: Age matters to survival. youngest and oldest most likely to survive. ~15-30 years old passengers more likely to die. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/alvinkan/opt/miniconda3/lib/python3.8/site-packages/numpy/lib/histograms.py:839: RuntimeWarning: invalid value encountered in greater_equal\n",
      "  keep = (tmp_a >= first_edge)\n",
      "/Users/alvinkan/opt/miniconda3/lib/python3.8/site-packages/numpy/lib/histograms.py:840: RuntimeWarning: invalid value encountered in less_equal\n",
      "  keep &= (tmp_a <= last_edge)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7ff0b0749ca0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd4AAADQCAYAAACp4wAHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAASX0lEQVR4nO3de7BdZX3G8e8jaBWxSjQwUaDRNqIWJcopRum0AsaJlxpalUK1k05t84+2SHUsaKfVcTrSGetlrDpmlBpvgNeSoQ6YiaGODreggCDXYhojkVy8a0eN/vrHWqceQ8jZOXufddbe+X5m9uy11t7rXc85k+SX911rvStVhSRJ6saDFjqAJEmHEguvJEkdsvBKktQhC68kSR2y8EqS1KHDuzzYqlWr6oorrujykJKk8ZGFDtCFTnu8u3fv7vJwkiT1jkPNkiR1yMIrSVKHLLySJHXIwitJUocsvJIkdcjCK0lShzq9j1eT6x0b7xxq//NWPnFESSSp3+zxSpLUIQuvJEkdsvBKktQhC68kSR2y8EqS1CGvalYveFW0pEOFPV5Jkjpk4ZUkqUMWXkmSOmThlSSpQxZeSZI6ZOGVJKlDFl5Jkjo00H28SbYCPwR+Aeytqqkki4BLgaXAVuCsqvru/MTUgXgPrCSNj4Pp8Z5WVcuraqpdPx/YVFXLgE3tuiRJOoBhhppXA+vb5fXAmUOnkSRpwg1aeAv4fJIbkqxttx1TVTsA2vej97djkrVJtiTZsmvXruETS5I0xgadq/nUqro3ydHAxiS3D3qAqloHrAOYmpqqOWSUJGliDNTjrap72/edwGeBU4D7kiwBaN93zldISZImxayFN8nDkzxiehl4HnALsAFY035tDXDZfIWUJGlSDDLUfAzw2STT3/94VV2R5HrgE0leCWwDXjZ/MSVJmgyzFt6qugc4aT/b9wBnzEcodWvY+4AlSYNz5ipJkjpk4ZUkqUMWXkmSOmThlSSpQxZeSZI6ZOGVJKlDFl5Jkjpk4ZUkqUMWXkmSOmThlSSpQxZeSZI6ZOGVJKlDFl5Jkjpk4ZUkqUMWXkmSOmThlSSpQxZeSZI6ZOGVJKlDAxfeJIcl+WqSy9v1RUk2JrmrfT9q/mJKkjQZDqbHey5w24z184FNVbUM2NSuS5KkAxio8CY5Fngh8IEZm1cD69vl9cCZI00mSdIEGrTH+07g9cAvZ2w7pqp2ALTvR482miRJk2fWwpvkRcDOqrphLgdIsjbJliRbdu3aNZcmJEmaGIP0eE8FXpxkK3AJcHqSjwL3JVkC0L7v3N/OVbWuqqaqamrx4sUjii1J0niatfBW1QVVdWxVLQXOBr5QVa8ANgBr2q+tAS6bt5SSJE2IYe7jvRBYmeQuYGW7LkmSDuDwg/lyVV0FXNUu7wHOGH0kSZImlzNXSZLUIQuvJEkdsvBKktQhC68kSR2y8EqS1CELryRJHTqo24l06Fixbd1I2rnm+LUjaUeSJoU9XkmSOmThlSSpQw41ayzMOvS9+dHdBJl22gXdHk/SxLDHK0lSh+zxSq2r79kz8Hev2Xvn/badt/KJo4wjaULZ45UkqUMWXkmSOmThlSSpQxZeSZI6ZOGVJKlDFl5Jkjpk4ZUkqUOzFt4kD01yXZKbktya5M3t9kVJNia5q30/av7jSpI03gbp8f4UOL2qTgKWA6uSrADOBzZV1TJgU7suSdK8SPLGtgN4c5IbkzxzBG2+OMlI6leSHw3yvVlnrqqqAqYbe3D7KmA18Jx2+3rgKuDvDzKnJEmzSvIs4EXAM6rqp0keAzxkwH0Pr6q9+/usqjYAG0aXdHYDTRmZ5DDgBuB3gPdU1bVJjqmqHQBVtSPJ0Q+w71pgLcDxxx8/mtQaG6N6ru9sDma6R0ljaQmwu6p+ClBVuwGSbAWmqmp3kingbVX1nCRvAh4LLAV2J/lt4C+r6tZ2v6uA1wJPBaaANwI3AU+oql8mOQK4A3gCcDzwHmAx8BPgr6vq9iSPBz5OU0uvGPQHGejiqqr6RVUtB44FTkly4qAHqKp1VTVVVVOLFy8edDdJkmb6PHBckjuTvDfJHw6wz8nA6qr6M+AS4CyAJEuAx1bVDdNfrKrv0xTe6Xb/CLiyqn4OrAP+pqpOBl4HvLf9zruA91XV7wHfHvQHOairmqvqezRDyquA+9rw0z/EzoNpS5KkQVXVj2gK6VpgF3Bpkr+YZbcNVfW/7fIngJe1y2cBn9zP9y8F/rRdPrs9xpHAs4FPJrkReD9N7xvgVODidvkjg/4sg1zVvDjJo9rlhwHPBW6nGRNf035tDXDZoAeVJOlgtaOvV1XVPwGvBl4C7OVXteyh++zy4xn7fgvYk+RpNMX1kv0cYgPw/CSLaIr8F9q2v1dVy2e8njwz1sH+HIP0eJcAm5PcDFwPbKyqy4ELgZVJ7gJWtuuSJI1ckhOSLJuxaTnwP8BWmiIJTSE+kEuA1wOPrKqv7fth26u+jmYI+fK20P8A+EaSl7U5kuSkdpcv0/SMAV4+6M8yyFXNNwNP38/2PcAZgx5IkqQhHAm8ux2B3QvcTTPs/GTgg0neAFw7SxufoimqbznAdy6lGYZ+zoxtLwfel+QfaO7suYTmfPC5wMeTnAt8etAfJM3dQt2YmpqqLVu2dHa8Q8U7Nt7/oezD6upq5HF1zfFr56Xd81Y+cV7alcZEFjpAF5wyUpKkDll4JUnq0EATaPTO5reOpp3TLhhNOz3iELEk9Zs9XkmSOmThlSSpQxZeSZI6ZOGVJGkASVYluSPJ3cM8SnA8L66SJB3Slp7/n28aZXtbL3zhAdtrn9L3HpqZGrcD1yfZUFVfP9hj2eOVJGl2pwB3V9U9VfUzmtmrVs+lIQuvJEmzexzwzRnr29ttB83CK0nS7PY3neWc5ly28EqSNLvtwHEz1o8F7p1LQxZeSZJmdz2wLMnjkzyE5nGAG+bSkFc1SxNk2CdV+XQkaf+qam+SVwNXAocBF1XVrXNpy8IrSRo7s93+Mx+q6nPA54Ztx6FmSZI6ZOGVJKlDFl5Jkjo0a+FNclySzUluS3JrknPb7YuSbExyV/t+1PzHlSRpvA3S490LvLaqngysAF6V5CnA+cCmqloGbGrXJUnSAcxaeKtqR1V9pV3+IXAbzTRZq4H17dfWA2fOU0ZJkibGQZ3jTbIUeDpwLXBMVe2ApjgDRz/APmuTbEmyZdeuXUPGlSRpYSS5KMnOJLcM087A9/EmORL4NPCaqvpBsr9pK++vqtYB6wCmpqbmNK+lJEm/5k2PfNNo2/v+IO19CPg34MPDHGqgHm+SB9MU3Y9V1WfazfclWdJ+vgTYOUwQSZL6rKq+CHxn2HYGuao5wAeB26rq7TM+2gCsaZfXAJcNG0aSpEk3yFDzqcCfA19LcmO77Q3AhcAnkrwS2Aa8bF4SSpI0QWYtvFX1Jfb/HEKAM0YbR5KkyebMVZIkdcjCK0nSAJJcDFwNnJBke3uq9aD5WEBJ0vgZ7Pafkaqqc0bRjoVXmoMV29YN3cY1x6+937ZhH2Qvqf8capYkqUMWXkmSOmThlSSpQxZeSZI65MVVfbH5rXPedcW2PSMMokPZKC7uOm/lE0eQRJpc9nglSeqQhVeSpA451CwJGM29yY23jagdaTLZ45UkqUMWXkmSOjTWQ81X3zPc1bzX7L3TKzAlSZ2yxytJUofGuscrjbPRXcwkaZzY45UkqUMWXkmSOjTrUHOSi4AXATur6sR22yLgUmApsBU4q6q+O38x+8sp9iRJB2OQHu+HgFX7bDsf2FRVy4BN7bokSZrFrIW3qr4IfGefzauB9e3yeuDM0caSJGkyzfUc7zFVtQOgfT/6gb6YZG2SLUm27Nq1a46HkyRpMsz7xVVVta6qpqpqavHixfN9OEmSem2uhfe+JEsA2vedo4skSdLkmusEGhuANcCF7ftlI0vUoRXb1sHmRw/Zxh6uOX7tiBJJkibdrD3eJBcDVwMnJNme5JU0BXdlkruAle26JEmaxaw93qo65wE+OmPEWSRJmnjO1SypV5yURpPOKSMlSeqQPV5JI3X1B1831P4r2vdhLlocttdsj1nzyR6vJEkdsvBKktQhh5olaR8OVWs+2eOVJKlDFl5Jkjp0yA81X33PnoWOwDs23smKbQufQ+qTFdvWjaQdp3RV39jjlSSpQxZeSZI6dMgPNY/CqIbEJI3euA5Ze2X15LLHK0lShyy8kiR1yMIrSVKHLLySJHXIi6skacRG8UxhTS57vJIkdcjCK0lSh4Yaak6yCngXcBjwgaq6cCSpJEnzb/NbR9POaReMpp1DxJx7vEkOA94DPB94CnBOkqeMKpgkSZNomKHmU4C7q+qeqvoZcAmwejSxJEmaTMMMNT8O+OaM9e3AM/f9UpK1wPRcaz9KcscQx3wMsHuI/ReCmbsxbpnHLS8c8pn/dTTNzG4kmf9uBEEG94ZR/Z6vqKpVI2in14YpvNnPtrrfhqp1wEgmS02ypaqmRtFWV8zcjXHLPG55wcxdMfPkG2aoeTtw3Iz1Y4F7h4sjSdJkG6bwXg8sS/L4JA8BzgY2jCaWJEmTac5DzVW1N8mrgStpbie6qKpuHVmy/RvH5++ZuRvjlnnc8oKZu2LmCZeq+52WlSRJ88SZqyRJ6pCFV5KkDo1F4U2yKskdSe5Ocv5C53kgSS5KsjPJLTO2LUqyMcld7ftRC5lxpiTHJdmc5LYktyY5t93e58wPTXJdkpvazG9ut/c287QkhyX5apLL2/VeZ06yNcnXktyYZEu7re+ZH5XkU0lub/9cP6vPmZOc0P5+p18/SPKanmc+r/27d0uSi9u/k73N20e9L7xjNjXlh4B9b/4+H9hUVcuATe16X+wFXltVTwZWAK9qf7d9zvxT4PSqOglYDqxKsoJ+Z552LnDbjPVxyHxaVS2fcY9m3zO/i2YShicBJ9H8vnubuaruaH+/y4GTgZ8An6WnmZM8DvhbYKqqTqS5sPZsepq3t6qq1y/gWcCVM9YvAC5Y6FwHyLsUuGXG+h3AknZ5CXDHQmc8QPbLgJXjkhk4AvgKzYxpvc5Mc5/7JuB04PJx+LMBbAUes8+23mYGfhP4Bu1Fo+OQeZ+czwO+3OfM/GrGwkU0d8Vc3ubuZd6+vnrf42X/U1M+boGyzMUxVbUDoH0/eoHz7FeSpcDTgWvpeeZ2yPZGYCewsap6nxl4J/B64JcztvU9cwGfT3JDO/Ur9DvzE4BdwL+3Q/ofSPJw+p15prOBi9vlXmauqm8BbwO2ATuA71fV5+lp3r4ah8I70NSUmrskRwKfBl5TVT9Y6DyzqapfVDM0dyxwSpITFzjSASV5EbCzqm5Y6CwH6dSqegbNaZ5XJfmDhQ40i8OBZwDvq6qnAz9mTIY820mIXgx8cqGzHEh77nY18HjgscDDk7xiYVONn3EovOM+NeV9SZYAtO87FzjPr0nyYJqi+7Gq+ky7udeZp1XV94CraM6r9znzqcCLk2yleYrX6Uk+Sr8zU1X3tu87ac47nkK/M28HtrcjIACfoinEfc487fnAV6rqvna9r5mfC3yjqnZV1c+BzwDPpr95e2kcCu+4T025AVjTLq+hOY/aC0kCfBC4rarePuOjPmdenORR7fLDaP4huJ0eZ66qC6rq2KpaSvPn9wtV9Qp6nDnJw5M8YnqZ5jzeLfQ4c1V9G/hmkhPaTWcAX6fHmWc4h18NM0N/M28DViQ5ov334wyaC9j6mrefFvok8yAv4AXAncB/A29c6DwHyHkxzXmPn9P87/uVwKNpLqq5q31ftNA5Z+T9fZph+5uBG9vXC3qe+WnAV9vMtwD/2G7vbeZ98j+HX11c1dvMNOdLb2pft07/vetz5jbfcmBL++fjP4CjxiDzEcAe4JEztvU2M/Bmmv/s3gJ8BPiNPuft48spIyVJ6tA4DDVLkjQxLLySJHXIwitJUocsvJIkdcjCK0lShyy80jxK8sdJKsmTFjqLpH6w8Erz6xzgSzQTZ0iShVeaL+0c2KfSTKRydrvtQUne2z7P9PIkn0vy0vazk5P8V/tQgiunp+CTNFksvNL8OZPm2bB3At9J8gzgT2geHflU4K9oHns5PWf2u4GXVtXJwEXAPy9AZknz7PCFDiBNsHNoHgcIzcMRzgEeDHyyqn4JfDvJ5vbzE4ATgY3NFLgcRjP9qKQJY+GV5kGSR9M89P7EJEVTSIvmKT/73QW4taqe1VFESQvEoWZpfrwU+HBV/VZVLa2q44BvALuBl7Tneo+heWgCwB3A4iT/P/Sc5HcXIrik+WXhlebHOdy/d/tpmoeHb6d5ssv7gWuB71fVz2iK9b8kuYnmSVHP7iytpM74dCKpY0mOrKoftcPR1wGnVvMsWUmHAM/xSt27PMmjgIcAb7HoSocWe7ySJHXIc7ySJHXIwitJUocsvJIkdcjCK0lShyy8kiR16P8AwqLxeLH42+AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 483.875x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "grid = sns.FacetGrid(train_df, hue='Survived', aspect=2)\n",
    "grid.map(plt.hist, 'Age', alpha=.5, bins=20)\n",
    "grid.add_legend()\n",
    "# youngest and oldest more likely to survive. 15-30 more likely not to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fill nulls for Age\n",
    "def Age_1(cols) :\n",
    "    Age=cols[0]\n",
    "    Pclass=cols[1] \n",
    "    \n",
    "    if pd.isnull(Age) :\n",
    "        if Pclass==1 :\n",
    "            return 38\n",
    "        elif Pclass==2:\n",
    "            return 29\n",
    "        else :\n",
    "            return 24\n",
    "    else :\n",
    "        return Age\n",
    "\n",
    "\n",
    "def Age_1(cols) :\n",
    "    Age=cols[0]\n",
    "    Pclass=cols[1] \n",
    "    \n",
    "    if pd.isnull(Age) :\n",
    "        if Pclass==1 :\n",
    "            return 42 \n",
    "        elif Pclass==2:\n",
    "            return 25\n",
    "        else :\n",
    "            return 22\n",
    "    else :\n",
    "        return Age\n",
    "    \n",
    "    \n",
    "test_df['Age']=test_df[['Age','Pclass']].apply(Age_1,axis=1)\n",
    "train_df['Age']=train_df[['Age','Pclass']].apply(Age_1,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create Age_bin column in train and test set\n",
    "def age_bin(dataset):\n",
    "    dataset['Age_bin'] = pd.cut(dataset['Age'], bins=[0,16,30,40,50,80], \n",
    "                                labels=[0,1,2,3,4])\n",
    "age_bin(train_df)\n",
    "age_bin(test_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Name/Title\n",
    "Name is not useful hence title column is created"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Mr          757\n",
       "Miss        260\n",
       "Mrs         197\n",
       "Master       61\n",
       "Dr            8\n",
       "Rev           8\n",
       "Col           4\n",
       "Ms            2\n",
       "Mlle          2\n",
       "Major         2\n",
       "Lady          1\n",
       "Dona          1\n",
       "Mme           1\n",
       "Sir           1\n",
       "Jonkheer      1\n",
       "Don           1\n",
       "Countess      1\n",
       "Capt          1\n",
       "Name: Name, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create title feature\n",
    "train_name = train_df['Name'].str.extract(' ([A-Za-z]+)\\.', expand=False)\n",
    "test_name = test_df['Name'].str.extract(' ([A-Za-z]+)\\.', expand=False)\n",
    "total_name = train_name.append(test_name)\n",
    "total_name.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Master</th>\n",
       "      <td>0.575000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Miss</th>\n",
       "      <td>0.702703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mr</th>\n",
       "      <td>0.156673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mrs</th>\n",
       "      <td>0.793651</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Rare</th>\n",
       "      <td>0.347826</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Survived\n",
       "Title           \n",
       "Master  0.575000\n",
       "Miss    0.702703\n",
       "Mr      0.156673\n",
       "Mrs     0.793651\n",
       "Rare    0.347826"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#create title column in train and test set\n",
    "def create_title(dataset):\n",
    "    dataset['Title'] = dataset['Name'].str.extract(' ([A-Za-z]+)\\.', expand=False)\n",
    "    \n",
    "    for title in dataset['Title']:\n",
    "        dataset['Title'] = dataset['Title'].replace(['Lady', 'Countess','Capt', 'Col',\n",
    "                                                    'Don', 'Dr', 'Major', 'Rev', 'Sir', 'Jonkheer', 'Dona'], 'Rare')\n",
    "        dataset['Title'] = dataset['Title'].replace('Mlle', 'Miss')\n",
    "        dataset['Title'] = dataset['Title'].replace('Ms', 'Miss')\n",
    "        dataset['Title'] = dataset['Title'].replace('Mme', 'Mrs')\n",
    "\n",
    "create_title(train_df)\n",
    "create_title(test_df)\n",
    "train_df[['Title','Survived']].groupby('Title').mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create Title_bin\n",
    "def title_bin(dataset):\n",
    "    dataset['Title_bin'] = dataset['Title'].map( {'Master': 0, 'Mr': 1, 'Miss': 2, 'Mrs': 3, 'Rare': 4} ).astype(int)\n",
    "\n",
    "title_bin(train_df)\n",
    "title_bin(test_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fare\n",
    "Observation: Fare negatively correlates with Pclass but not to the extent to remove it. Lowest fare bin more likely to die."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7ff0903ace20>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd4AAADQCAYAAACp4wAHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAARdUlEQVR4nO3df6zddX3H8efLguDEIUghHW1GXaoTzATpGI7EiYo001mWBdcppmYk+Acu6OZM6/4QszDZkjlNlMVGiXX+qHVq6JhBmwrRLQgUBKEg0AlCB6MF4w+MY7a898f5Vk7bW+5p7zmfe889z0dyc77fz/l8P+d9Pim87vd7vz9SVUiSpDaeM9sFSJI0SQxeSZIaMnglSWrI4JUkqSGDV5Kkho6Y7QIAVqxYUdddd91slyFJmj2Z7QJamRN7vI8//vhslyBJUhNzInglSZoUBq8kSQ0ZvJIkNWTwSpLUkMErSVJDBq8kSQ3Niet4h+WfNt83lHHec95LhjKOJEn7c49XkqSGDF5JkhoyeCVJasjglSSpoYGDN8mCJN9Ncm23fnySzUnu716P6+u7Nsn2JPcmOX8UhUuSNI4OZY/3MuCevvU1wJaqWgZs6dZJciqwCjgNWAFclWTBcMqVJGm8DRS8SRYDbwQ+2de8EljfLa8HLuhr31BVT1XVA8B24KyhVCtJ0pgbdI/3I8D7gKf72k6qqkcButcTu/aTgYf7+u3o2vaR5JIkW5Ns3bVr16HWLUnSWJo2eJO8CdhZVbcOOOZUDzOuAxqq1lXV8qpavnDhwgGHliRpvA1y56pzgDcn+UPgaODXk3wWeCzJoqp6NMkiYGfXfwewpG/7xcAjwyxakqRxNe0eb1WtrarFVXUKvZOmvllVFwGbgNVdt9XANd3yJmBVkqOSLAWWATcPvXJJksbQTO7VfCWwMcnFwEPAhQBVtS3JRuBuYDdwaVXtmXGlkiTNA4cUvFV1A3BDt/wE8LqD9LsCuGKGtUmSNO945ypJkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGDF5JkhoyeCVJasjglSSpIYNXkqSGpg3eJEcnuTnJHUm2Jflg1358ks1J7u9ej+vbZm2S7UnuTXL+KL+AJEnjZJA93qeA11bVK4DTgRVJzgbWAFuqahmwpVsnyanAKuA0YAVwVZIFI6hdkqSxM23wVs+T3eqR3U8BK4H1Xft64IJueSWwoaqeqqoHgO3AWcMsWpKkcTXQ33iTLEhyO7AT2FxVNwEnVdWjAN3riV33k4GH+zbf0bXtP+YlSbYm2bpr164ZfAVJksbHQMFbVXuq6nRgMXBWkpc/S/dMNcQUY66rquVVtXzhwoUDFStJ0rg7pLOaq+rHwA30/nb7WJJFAN3rzq7bDmBJ32aLgUdmWqgkSfPBIGc1L0zywm75ecDrge8Dm4DVXbfVwDXd8iZgVZKjkiwFlgE3D7luSZLG0hED9FkErO/OTH4OsLGqrk1yI7AxycXAQ8CFAFW1LclG4G5gN3BpVe0ZTfmSJI2XaYO3qr4HnDFF+xPA6w6yzRXAFTOuTpKkecY7V0mS1JDBK0lSQwavJEkNGbySJDVk8EqS1JDBK0lSQwavJEkNGbySJDVk8EqS1JDBK0lSQwavJEkNGbySJDVk8EqS1JDBK0lSQwavJEkNGbySJDVk8EqS1JDBK0lSQwavJEkNHTHbBQzT2Q+tG85A179o5mOcu3bmY0iSAEjyN8BbgT3A08A7q+qmGY75ZuDUqrpyCPU9WVXHDNJ3XgWvJGn+SfIq4E3AK6vqqSQnAM8dcNsjqmr3VO9V1SZg0/AqHYyHmiVJc90i4PGqegqgqh6vqkeSPNiFMEmWJ7mhW748ybok3wA+k+SmJKftHSzJDUnOTPKOJB9Lcmw31nO6938tycNJjkzyW0muS3Jrkm8n+e2uz9IkNya5JcnfHsqXMXglSXPdN4AlSe5LclWSPxhgmzOBlVX1VmAD8BaAJIuA36iqW/d2rKqfAHcAe8f9I+DrVfVLYB3wF1V1JvBe4Kquz0eBf66q3wX+51C+jMErSZrTqupJekF6CbAL+GKSd0yz2aaq+kW3vBG4sFt+C/ClKfp/EfjTbnlV9xnHAL8PfCnJ7cAn6O19A5wDfKFb/pdD+T7+jVeSNOdV1R7gBuCGJHcCq4HdPLMDefR+m/y8b9v/TvJEkt+hF67vnOIjNgEfSnI8vZD/JvB84MdVdfrByjqc7+IeryRpTkvy0iTL+ppOB34IPEgvJAH+ZJphNgDvA46tqjv3f7Pbq76Z3iHka6tqT1X9FHggyYVdHUnyim6T/6S3ZwzwtkP5PgavJGmuOwZYn+TuJN8DTgUuBz4IfDTJt+ldZvRs/pVeUG58lj5fBC7qXvd6G3BxkjuAbcDKrv0y4NIktwDHHsqXSdVh7SkP1fLly2vr1q0zHufGT713CNXAq17sdbyS1Fhmu4BWpt3jTbIkyfVJ7kmyLcllXfvxSTYnub97Pa5vm7VJtie5N8n5o/wCkiSNk0EONe8G/qqqXgacTW/X+lRgDbClqpYBW7p1uvdWAacBK4CrkiwYRfGSJI2baYO3qh6tqtu65Z8B9wAn0zvOvb7rth64oFteCWyoqqeq6gFgO3DWkOuWJGksHdLJVUlOAc4AbgJOqqpHoRfOwIldt5OBh/s229G17T/WJUm2Jtm6a9euwyhdkqTxM3DwdhcSfxl4d3eK9UG7TtF2wBlcVbWuqpZX1fKFCxcOWoYkSWNtoOBNciS90P1cVX2la36su/XW3ltw7ezadwBL+jZfDDwynHIlSRpvg5zVHOBTwD1V9eG+tzbRu3MI3es1fe2rkhyVZCmwjN5FyZIkja0kK7qrdbYnWXO44wxyy8hzgLcDd3b3qgR4P3AlsDHJxcBDdPfBrKptSTYCd9M7I/rS7lZfkiTN2Clr/v3yYY734JVvnHa87uqcjwPn0Tuye0uSTVV196F+3rTBW1X/wcEvbH7dQba5ArjiUIuRJGmOOgvYXlU/AEiygd5VPIccvN4yUpKk6Q10xc4gDF5JkqY30BU7gzB4JUma3tCu2DF4JUma3i3AsiRLkzyX3q2RNx3OQIOc1SxJ0kSrqt1J3gV8HVgAXF1V2w5nLINXkjRWBrn8ZxSq6mvA12Y6joeaJUlqyOCVJKkhg1eSpIYMXkmSGjJ4JUlqyOCVJKkhg1eSpGkkuTrJziR3zXQsr+OVJI2Xy4+9fLjj/WSQ8T4NfAz4zEw/zj1eSZKmUVXfAn40jLEMXkmSGjJ4JUlqyOCVJKkhg1eSpIYMXkmSppHkC8CNwEuT7Ehy8eGO5eVEkqTxMtjlP0NVVX82rLHc45UkqSGDV5KkhgxeSZIaMnglSWrI4JUkqSGDV5KkhgxeSZIamjZ4p3oGYZLjk2xOcn/3elzfe2uTbE9yb5LzR1W4JEnjaJA93k8DK/ZrWwNsqaplwJZunSSnAquA07ptrkqyYGjVSpI05qYN3oM8g3AlsL5bXg9c0Ne+oaqeqqoHgO3AWcMpVZKk8Xe4f+M9qaoeBeheT+zaTwYe7uu3o2s7QJJLkmxNsnXXrl2HWYYkSeNl2CdXZYq2mqpjVa2rquVVtXzhwoVDLkOSpLnpcIP3sSSLALrXnV37DmBJX7/FwCOHX54kSfPL4QbvJmB1t7wauKavfVWSo5IsBZYBN8+sREmS5o9pHwvYPYPwNcAJSXYAHwCuBDZ2zyN8CLgQoKq2JdkI3A3sBi6tqj0jql2SpLEzbfA+yzMIX3eQ/lcAV8ykKEmS5qtpg3cS3fiDJ2Y8xnd238d7znvJEKqRJM0n3jJSkqSGDF5JkhoyeCVJasjglSSpIU+uGpGzH1oH179otsvoOXftbFcgSeq4xytJUkMGryRJDRm8kiQ1ZPBKktSQwStJUkMGryRJDRm8kiQ1ZPBKktSQwStJUkMGryRJDRm8kiQ1ZPBKktSQwStJUkMGryRJDRm8kiQ1ZPBKktTQEbNdwHx24w+eGMo4r3rxi2Y2wPUfGkodQ3Hu2tmuQJJmlXu8kiQ1ZPBKktSQwStJUkMGryRJDRm8kiQ1NLKzmpOsAD4KLAA+WVVXjuqzpMPi2d6SZsFIgjfJAuDjwHnADuCWJJuq6u5RfN58N6zLkuaC7+y+71fL7znvJbNYyRzjLwHSxBjVHu9ZwPaq+gFAkg3ASsDgnXBnP7TumZXrZ3h98pDNtV9wZnz9tuY3f1kbW6MK3pOBh/vWdwC/198hySXAJd3qk0nuHcLnngA8PoRx5gvn40DOyb6mmI/3z0ohc4j/RvY1wHwM5d/MdVW1YhgDzXWjCt5M0Vb7rFStA9ZN0e/wPzTZWlXLhznmOHM+DuSc7Mv5OJBzsi/nY/hGdVbzDmBJ3/pi4JERfZYkSWNjVMF7C7AsydIkzwVWAZtG9FmSJI2NkRxqrqrdSd4FfJ3e5URXV9W2UXzWfoZ66HoecD4O5Jzsy/k4kHOyL+djyFJV0/eSJElD4Z2rJElqyOCVJKmheRG8SVYkuTfJ9iRrZrueVpJcnWRnkrv62o5PsjnJ/d3rcX3vre3m6N4k589O1aOTZEmS65Pck2Rbksu69omckyRHJ7k5yR3dfHywa5/I+eiXZEGS7ya5tluf2DlJ8mCSO5PcnmRr1zax89FEVY31D72Tt/4LeDHwXOAO4NTZrqvRd3818Ergrr62fwDWdMtrgL/vlk/t5uYoYGk3Zwtm+zsMeT4WAa/sll8A3Nd974mcE3rX0x/TLR8J3AScPanzsd/c/CXweeDabn1i5wR4EDhhv7aJnY8WP/Nhj/dXt6esqv8D9t6ect6rqm8BP9qveSWwvlteD1zQ176hqp6qqgeA7fTmbt6oqker6rZu+WfAPfTuojaRc1I9T3arR3Y/xYTOx15JFgNvBD7Z1zzRczIF52OE5kPwTnV7ypNnqZa54KSqehR6QQSc2LVP1DwlOQU4g95e3sTOSXdI9XZgJ7C5qiZ6PjofAd4HPN3XNslzUsA3ktza3coXJns+Rm5kjwVsaNrbUwqYoHlKcgzwZeDdVfXTZKqv3us6Rdu8mpOq2gOcnuSFwFeTvPxZus/7+UjyJmBnVd2a5DWDbDJF27yaE+CcqnokyYnA5iTff5a+kzAfIzcf9ni9PeW+HkuyCKB73dm1T8Q8JTmSXuh+rqq+0jVP9JwAVNWPgRuAFUz2fJwDvDnJg/T+LPXaJJ9lguekqh7pXncCX6V36Hhi56OF+RC83p5yX5uA1d3yauCavvZVSY5KshRYBtw8C/WNTHq7tp8C7qmqD/e9NZFzkmRht6dLkucBrwe+z4TOB0BVra2qxVV1Cr3/V3yzqi5iQuckyfOTvGDvMvAG4C4mdD5aGftDzTV7t6ecdUm+ALwGOCHJDuADwJXAxiQXAw8BFwJU1bYkG+k9E3k3cGl3GHI+OQd4O3Bn93dN6D2vbFLnZBGwPskCer9kb6yqa5PcyGTOx7OZ1H8jJ9H7EwT08uDzVXVdkluYzPlowltGSpLU0Hw41CxJ0tgweCVJasjglSSpIYNXkqSGDF5Jkhoa+8uJpLkuyR7gzr6mC6rqwVkqR9Is83IiacSSPFlVxxziNqH33+fT03aWNFY81Cw1luSYJFuS3NY9B3Vl135Kes8Svgq4DViS5K+T3JLke3ufpytpvBm80ug9r3vI+O1Jvgr8L/DHVfVK4FzgH/PMkxxeCnymqs7olpfRu3fu6cCZSV7dvnxJw+TfeKXR+0VVnb53pXuQw991Ifo0vceqndS9/cOq+k63/Ibu57vd+jH0gvhbLYqWNBoGr9Te24CFwJlV9cvuSTlHd+/9vK9fgA9V1Sca1ydphDzULLV3LL1nwv4yybnAbx6k39eBP++eL0ySk7tnpkoaY+7xSu19Dvi3JFuB2+k9qu8AVfWNJC8Dbuz+BPwkcBHPPBtV0hjyciJJkhryULMkSQ0ZvJIkNWTwSpLUkMErSVJDBq8kSQ0ZvJIkNWTwSpLU0P8DoxT77BwmSMYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 483.875x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# fill null fares in test set based on Pclass\n",
    "pclass_fare = test_df[['Pclass','Fare']].groupby('Pclass').median()\n",
    "fare_dict = pclass_fare.to_dict()['Fare']\n",
    "test_df['Fare'].fillna(fare_dict[test_df['Pclass'].iloc[0]], inplace=True)\n",
    "\n",
    "# visualise survival rates\n",
    "grid = sns.FacetGrid(train_df, hue='Survived', aspect=2)\n",
    "grid.map(plt.hist, 'Fare', bins=10, alpha = 0.5)\n",
    "grid.add_legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare_bin</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>223</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>222</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Survived\n",
       "Fare_bin          \n",
       "0              223\n",
       "1              224\n",
       "2              222\n",
       "3              222"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#create fare_bin column in train and test set\n",
    "def fare_bin(dataset):\n",
    "    dataset['Fare_bin'] = pd.qcut(dataset['Fare'], q=4, labels=[0,1,2,3])\n",
    "fare_bin(train_df)\n",
    "fare_bin(test_df)\n",
    "train_df[['Fare_bin','Survived']].groupby('Fare_bin').count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Sex_bin</th>\n",
       "      <th>Title_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.005007</td>\n",
       "      <td>-0.035144</td>\n",
       "      <td>0.036496</td>\n",
       "      <td>-0.057527</td>\n",
       "      <td>-0.001652</td>\n",
       "      <td>0.012658</td>\n",
       "      <td>-0.042939</td>\n",
       "      <td>0.017598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>-0.005007</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.338481</td>\n",
       "      <td>-0.037798</td>\n",
       "      <td>-0.035322</td>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.257307</td>\n",
       "      <td>0.543351</td>\n",
       "      <td>0.398307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.035144</td>\n",
       "      <td>-0.338481</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.436993</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>-0.549500</td>\n",
       "      <td>-0.131900</td>\n",
       "      <td>-0.246588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.036496</td>\n",
       "      <td>-0.037798</td>\n",
       "      <td>-0.436993</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.243914</td>\n",
       "      <td>-0.166945</td>\n",
       "      <td>0.138049</td>\n",
       "      <td>-0.074819</td>\n",
       "      <td>0.222765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.057527</td>\n",
       "      <td>-0.035322</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>-0.243914</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>0.114631</td>\n",
       "      <td>-0.008921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>-0.001652</td>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>-0.166945</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>0.245489</td>\n",
       "      <td>0.128302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.012658</td>\n",
       "      <td>0.257307</td>\n",
       "      <td>-0.549500</td>\n",
       "      <td>0.138049</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.182333</td>\n",
       "      <td>0.146020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex_bin</th>\n",
       "      <td>-0.042939</td>\n",
       "      <td>0.543351</td>\n",
       "      <td>-0.131900</td>\n",
       "      <td>-0.074819</td>\n",
       "      <td>0.114631</td>\n",
       "      <td>0.245489</td>\n",
       "      <td>0.182333</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.751016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Title_bin</th>\n",
       "      <td>0.017598</td>\n",
       "      <td>0.398307</td>\n",
       "      <td>-0.246588</td>\n",
       "      <td>0.222765</td>\n",
       "      <td>-0.008921</td>\n",
       "      <td>0.128302</td>\n",
       "      <td>0.146020</td>\n",
       "      <td>0.751016</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             PassengerId  Survived    Pclass       Age     SibSp     Parch  \\\n",
       "PassengerId     1.000000 -0.005007 -0.035144  0.036496 -0.057527 -0.001652   \n",
       "Survived       -0.005007  1.000000 -0.338481 -0.037798 -0.035322  0.081629   \n",
       "Pclass         -0.035144 -0.338481  1.000000 -0.436993  0.083081  0.018443   \n",
       "Age             0.036496 -0.037798 -0.436993  1.000000 -0.243914 -0.166945   \n",
       "SibSp          -0.057527 -0.035322  0.083081 -0.243914  1.000000  0.414838   \n",
       "Parch          -0.001652  0.081629  0.018443 -0.166945  0.414838  1.000000   \n",
       "Fare            0.012658  0.257307 -0.549500  0.138049  0.159651  0.216225   \n",
       "Sex_bin        -0.042939  0.543351 -0.131900 -0.074819  0.114631  0.245489   \n",
       "Title_bin       0.017598  0.398307 -0.246588  0.222765 -0.008921  0.128302   \n",
       "\n",
       "                 Fare   Sex_bin  Title_bin  \n",
       "PassengerId  0.012658 -0.042939   0.017598  \n",
       "Survived     0.257307  0.543351   0.398307  \n",
       "Pclass      -0.549500 -0.131900  -0.246588  \n",
       "Age          0.138049 -0.074819   0.222765  \n",
       "SibSp        0.159651  0.114631  -0.008921  \n",
       "Parch        0.216225  0.245489   0.128302  \n",
       "Fare         1.000000  0.182333   0.146020  \n",
       "Sex_bin      0.182333  1.000000   0.751016  \n",
       "Title_bin    0.146020  0.751016   1.000000  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#check correlation between fare and other columns\n",
    "train_df.corr(method ='pearson') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Embarked\n",
    "Observation: Those embarked from port C more likely to survive"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>C</th>\n",
       "      <td>0.553571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Q</th>\n",
       "      <td>0.389610</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>S</th>\n",
       "      <td>0.336957</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Survived\n",
       "Embarked          \n",
       "C         0.553571\n",
       "Q         0.389610\n",
       "S         0.336957"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove rows with null 'Embarked'\n",
    "train_df = train_df[pd.notnull(train_df['Embarked'])]\n",
    "\n",
    "train_df[['Embarked','Survived']].groupby('Embarked').mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-20-c1a5d6e4f0e1>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['Embarked_bin'] = dataset['Embarked'].map( {'C': 0, 'Q': 1, 'S': 2} ).astype(int)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked_bin</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>644</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Survived\n",
       "Embarked_bin          \n",
       "0                  168\n",
       "1                   77\n",
       "2                  644"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#create Embarked_bin\n",
    "def embarked_bin(dataset):\n",
    "    dataset['Embarked_bin'] = dataset['Embarked'].map( {'C': 0, 'Q': 1, 'S': 2} ).astype(int)\n",
    "\n",
    "embarked_bin(train_df)\n",
    "embarked_bin(test_df)\n",
    "\n",
    "train_df[['Embarked_bin','Survived']].groupby('Embarked_bin').count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Family Size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-21-0a17978557fa>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  train_df['Family_size'] = train_df['SibSp'] + train_df['Parch'] + 1\n"
     ]
    }
   ],
   "source": [
    "# create familysize feature\n",
    "train_df['Family_size'] = train_df['SibSp'] + train_df['Parch'] + 1\n",
    "test_df['Family_size'] = test_df['SibSp'] + test_df['Parch'] + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Family Survival\n",
    "\n",
    "The belief is that if someone in the family survives, the whole family has a higher chance of survival"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of passengers with family survival information: 369\n"
     ]
    }
   ],
   "source": [
    "# create df_full to apply transformation to both train and test sets\n",
    "full_df = train_df.append(test_df, sort=False)\n",
    "\n",
    "# extract the last name from the 'Name' column (using the full_data)\n",
    "for name in full_df['Name']:\n",
    "    full_df['Last Name'] = full_df['Name'].str.extract('([A-Za-z]+)\\,', expand=False)\n",
    "\n",
    "# create family_survival column with default value\n",
    "default_survival_value = 0.5\n",
    "full_df['Family_survival'] = default_survival_value\n",
    "\n",
    "# assign survival value based on identified families and known survivors in family\n",
    "for grp, grp_df in full_df[['Survived','Name', 'Last Name', 'Fare', 'Ticket', 'PassengerId',\n",
    "                       'SibSp', 'Parch', 'Age']].groupby(['Last Name', 'Fare', 'Ticket']):\n",
    "\n",
    "    #if a family is found\n",
    "    if (len(grp_df) > 1):\n",
    "        for i, row in grp_df.iterrows():\n",
    "            smax = grp_df.drop(i)['Survived'].max()\n",
    "            smin = grp_df.drop(i)['Survived'].min()\n",
    "            passID = row['PassengerId']\n",
    "\n",
    "            #if at one person survived in the family\n",
    "            if (smax == 1.0):\n",
    "                full_df.loc[full_df['PassengerId'] == passID, 'Family_survival'] = 1\n",
    "                \n",
    "            #if no one survived in the family    \n",
    "            elif (smin==0.0):\n",
    "                full_df.loc[full_df['PassengerId'] == passID, 'Family_survival'] = 0\n",
    "\n",
    "# see number of passengers with family survival info\n",
    "print(\"Number of passengers with family survival information:\", \n",
    "      full_df.loc[full_df['Family_survival']!=0.5].shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-23-f88ab419217a>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  train_df['Family_survival'] = full_df['Family_survival'][:len(train_df)]\n"
     ]
    }
   ],
   "source": [
    "# add family survival column to train and test set by splicing\n",
    "train_df['Family_survival'] = full_df['Family_survival'][:len(train_df)]\n",
    "test_df['Family_survival'] = full_df['Family_survival'][len(train_df):]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cabin\n",
    "Observation: too many null values to use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1307 entries, 0 to 417\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   Ticket  1307 non-null   object\n",
      " 1   Cabin   293 non-null    object\n",
      "dtypes: object(2)\n",
      "memory usage: 30.6+ KB\n"
     ]
    }
   ],
   "source": [
    "full_df[['Ticket', 'Cabin']].info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop cabin column\n",
    "test_df = test_df.drop(['Cabin'], axis=1)\n",
    "train_df = train_df.drop(['Cabin'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Sex_bin</th>\n",
       "      <th>Age_bin</th>\n",
       "      <th>Title</th>\n",
       "      <th>Title_bin</th>\n",
       "      <th>Fare_bin</th>\n",
       "      <th>Embarked_bin</th>\n",
       "      <th>Family_size</th>\n",
       "      <th>Family_survival</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>Q</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Mrs</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>Q</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>Wirz, Mr. Albert</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315154</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Mrs. Alexander (Helga E Lindqvist)</td>\n",
       "      <td>female</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mrs</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                          Name     Sex  \\\n",
       "0          892       3                              Kelly, Mr. James    male   \n",
       "1          893       3              Wilkes, Mrs. James (Ellen Needs)  female   \n",
       "2          894       2                     Myles, Mr. Thomas Francis    male   \n",
       "3          895       3                              Wirz, Mr. Albert    male   \n",
       "4          896       3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)  female   \n",
       "\n",
       "    Age  SibSp  Parch   Ticket     Fare Embarked  Sex_bin Age_bin Title  \\\n",
       "0  34.5      0      0   330911   7.8292        Q        0       2    Mr   \n",
       "1  47.0      1      0   363272   7.0000        S        1       3   Mrs   \n",
       "2  62.0      0      0   240276   9.6875        Q        0       4    Mr   \n",
       "3  27.0      0      0   315154   8.6625        S        0       1    Mr   \n",
       "4  22.0      1      1  3101298  12.2875        S        1       1   Mrs   \n",
       "\n",
       "   Title_bin Fare_bin  Embarked_bin  Family_size  Family_survival  \n",
       "0          1        0             1            1              0.5  \n",
       "1          3        0             2            2              0.5  \n",
       "2          1        1             1            1              0.5  \n",
       "3          1        1             2            1              0.5  \n",
       "4          3        1             2            3              1.0  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select features for model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "selected = ['Family_survival','Family_size','Pclass','Sex_bin','Age_bin','Title_bin','Fare_bin','Embarked_bin']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Further split training set into train and test set for prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((711, 8), (178, 8), (711,), (178,))"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# split train_df further into subsets for training and testing\n",
    "features = train_df[selected]\n",
    "target = train_df[\"Survived\"]\n",
    "X_train,X_test,y_train,y_test = train_test_split(features,target,test_size=0.2)\n",
    "X_train.shape,X_test.shape,y_train.shape,y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8888888888888888\n",
      "0.8539325842696629\n",
      "0.8368348982785602\n"
     ]
    }
   ],
   "source": [
    "# define classifier\n",
    "model = RandomForestClassifier(criterion=\"gini\", max_features=5,  \n",
    "                               min_samples_split=8, n_estimators=400)\n",
    "\n",
    "# fit classifier on train data\n",
    "model.fit(X_train, y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "\n",
    "# print accuracy scores\n",
    "print(model.score(X_train, y_train))\n",
    "print(accuracy_score(y_pred,y_test))\n",
    "print(cross_val_score(model,X_train,y_train,cv=10,scoring='accuracy').mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8888888888888888\n",
      "0.7865168539325843\n",
      "0.8410406885758999\n"
     ]
    }
   ],
   "source": [
    "# logistic regression\n",
    "logreg = LogisticRegression()\n",
    "logreg.fit(X_train, y_train)\n",
    "y_pred = logreg.predict(X_test)\n",
    "\n",
    "print(model.score(X_train, y_train))\n",
    "print(accuracy_score(y_pred,y_test))\n",
    "print(cross_val_score(model,X_train,y_train,cv=10,scoring='accuracy').mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsSklEQVR4nO3dd5hcZfn/8fcnBQg1QFaFFAKhKz30CKygKTSBCIh0MUbQHwjSRBSUIkVQpIReBMFQ5UtRQZYAAoEECC2UJIAE0yBAQgiGwP374zlrJsPO7mx2Z2dm5/O6rnPtzJxnztxzdnfuOU9VRGBmZrWrS7kDMDOz8nIiMDOrcU4EZmY1zonAzKzGORGYmdU4JwIzsxrnRFBjJIWktUtw3I8krVWC454m6cZ2Pub9kg5pZv91ks5ow/EPlfTYkj6/la/1v99nW+OuJO35XkrxN9TZOBFUIUmDJD0u6UNJsyX9S9KWHfj6D0s6IvexiFg+IqZ0VAxtERFDI+J6aPuHtqT+2Ydxt/aLcLHjnybp0yzRNm4flOK1lkT23mfkvn9J3STNlFTUIKWOTJzWNCeCKiNpReAe4I/AKkBv4HTgv+WMy0rqL1mibdx6ljugPB8AQ3PuDwPeL08otiScCKrPugARcXNEfBYR8yPiHxHxfGMBSYdLmijpfUl/l7RGUweStLSk8yX9O/tWN0pSj5z9e0p6TtIcSZMlDZF0JvB14OLs2+nFWdn8KopLJN0raa6ksZIG5Bz3W5Jeza5oLpU0Jv8Ko0C83SXdLOl2SUvl7VtT0geSumT3r5I0M2f/jZKOyW4/LOkISRsAo4Btm/imvXKh+PM8kv38IDvGtjmveX72O3hD0tCcx1eSdLWkaZLekXSGpK4tvf8i9ZL0QBb3mNzfvaTtJD2dnfenJW2XPV4v6YWccg9Keirn/mOSvt3Ma/4JODjn/sHADbkFCr3nJf0dFHov2b41s/c+V9IDQK/iTl0NiwhvVbQBKwLvAdeTvoWtnLf/28AkYAOgG/AL4PGc/QGsnd3+PXA36cpiBeD/gLOzfVsBHwLfJH1h6A2sn+17GDgi73Vzj3sdMDs7RjfgJuCWbF8vYA6wd7bvaODT/OPlHPc04EagB3BvduyuBcr+G9giu/0qMAXYIGffZvnxA4cCj+Udp2D8Tbxm/+y9d8t57NDsPf0A6Ar8CPgPoGz/XcDlwHLAl4CngB829/6b+XvIP+9zgR2ApYE/NL637Hf8PnBQ9p6+m91fFVgGmJ/9broB07N4V8jO+3xg1WZe/2vADKBnts3IHouccgXfc2t/B829l2z/E8AF2TnYITsnBc+ht/AVQbWJiDnAINI/4JXALEl3S/pyVuSHpA/ziRGxEDgL2DT/qkCSSB9UP42I2RExNyu7f1bk+8A1EfFARHweEe9ExCutCPWOiHgqi+EmYNPs8WHASxFxR7bvItIHT3NWBP4GTAYOi4jPCpQbA+wo6SvZ/duy+2tmx5jQDvEX662IuDKL9XpgNeDL2e9pKHBMRMyLiJnAhSw6703ZN7vaadwamil7b0Q8EhH/BU4hfdPuC+wKvB4Rf4qIhRFxM/AKsHtEfAKMI31oDgSeBx4Dtge2yZ73XjOv+QnpS8R+2fu4O3sMgCV8z1D4d1DwvUjqB2wJnBoR/42IR7LYrBklaeCy0oqIiaRvUUhan/SN+fekb0ZrAH+Q9Lucp4j0jf6tnMfqgGWB8Skn/K9cYxVFX+C+NoSZ++H+MbB8dnt14O2c9xKSprZwrG2A7sB3I/vKV8AYYA9gKqnK5mHSt8ZPgEcj4vN2iL/Vz4+Ij7NzvDzp22x3YFrOee9CzjlpwuiIOLDI1809tx9Jmk0656uz+O+f7H7v7PYYYCfSuRtD+oa9I6ntaUwRr3sDcDbpb+jEvH1r0Pr3DM3/DRV6L6sD70fEvLx9fVt+C7XLiaDKRcQrkq4jXQlA+uc6MyJuauGp75Iu+b8aEe80sf9toFC9eFumrJ0G9Gm8k12Z9ClcHIB/kL6l/lPSThExo0C5McB5LPowe4xU//wJhT/M2jr9bmuf/zbpw7VX9k23vf3vA09SY+L5T7bltxX1I11pQTo/vyNVof2WlAiuzGK9pIjXfZR01ROk8577t9PSe27tOWzuvUwjtS0sl5MM+i3Ba9QUVw1VGUnrSzpOUp/sfl/SlcCTWZFRwMmSvprtX0nSd/KPk307vhK4UNKXsrK9JQ3OilwNHCZpZ0ldsn3rZ/tmAEs6ZuBeYCNJ31bqcngU8JUWnkNEnAv8mZQMmmz8i4jXScntQOCRrBptBrAPhRPBDKBPfuNzK8wCPqfI8xER00iJ7XeSVszO7QBJOy7h6+cbptS9eCngN8DYiHibdHW3rqQDlLp37gdsSOqBBvA4sB6pTv6piHiJ9GG7NYsaxJt7XwHsDuyRf9VWxHtu7e+g4HuJiLdI1VynS1pK0qAsLmuGE0H1mUv65xwraR4pAbwIHAcQEXcC5wC3SJqT7Rta4FgnkhqWn8zKPkj6MCAingIOI9Xlfkj6IG38FvYHYHjWI+ai1gQfEe8C3wHOJTV6b0j6x22x+2tE/IbU6PigpFUKFBsDvBcR/865L+DZAuUfAl4Cpkt6t8i3kRvTx8CZwL+y+vttinjawcBSwMukb963kb5NF7KfFh9H8FFj8m7Cn4FfkRpatwC+l8X5HrAb6e/kPeAEYLfs90H27fkZUvvNguxYT5DaOmZShIh4KUsgrX3PrfodtPRegANI/yOzSefihqaOY4somq1yNSstpe6eU4HvRURzjaBmViK+IrAOJ2mwpJ6SlgZ+TvrG/mQLTzOzEnEisHLYltQV9F1S/e23I2J+eUMyq12uGjIzq3G+IjAzq3FVN46gV69e0b9//3KHYWZWVcaPH/9uRNQ1ta/qEkH//v0ZN25cucMwM6sqkvJHY/+Pq4bMzGqcE4GZWY1zIjAzq3FOBGZmNc6JwMysxnX6RHDuudCQN4NNQ0N63MzMaiARbLkl7LvvomTQ0JDub7lleeMyM6sUVTeOoLXq62H0aNhnH1h3XZg8Od2vry93ZGZmlaHTXxFA+tAfPBjGjoVvfctJwMwsV00kgoYGePBB6NoV7rzzi20GZma1rNMngsY2gdGjYZddoFevxdsMzMxqXadPBE8/vahNYOhQePtt+MMf0uNmZlYDieCEExa1CQzNVu794IP0uJmZ1UAiyLXOOrDWWnD//eWOxMysctRUIpBg2DD45z/hk0/KHY2ZWWWoqUQAqXpo/nx45JFyR2JmVhlKlggk9ZXUIGmipJckHd1M2S0lfSZpeKniabTTTrD00q4eMjNrVMorgoXAcRGxAbANcJSkDfMLSeoKnAP8vYSx/M+yy6Zk4ERgZpaULBFExLSIeCa7PReYCPRuouhPgNuBmaWKJd/QofDqqzBlSke9oplZ5eqQNgJJ/YHNgLF5j/cG9gJGtfD8EZLGSRo3a9asNsfT2I3UVwVmZh2QCCQtT/rGf0xEzMnb/XvgxIj4rLljRMQVETEwIgbW1dW1OaZ11oEBA5wIzMygxLOPSupOSgI3RcQdTRQZCNwiCaAXMEzSwoi4q7RxpauCq69O3UiXWaaUr2ZmVtlK2WtIwNXAxIi4oKkyEbFmRPSPiP7AbcCRpU4CjdyN1MwsKWXV0PbAQcA3JD2XbcMkjZQ0soSvWxR3IzUzS0pWNRQRjwFqRflDSxVLUxq7kd53H1x4YUe+splZZam5kcW5hg2D115zN1Izq201nQjcjdTMrMYTgbuRmpnVeCKAdFXw0EOejdTMapcTgbuRmlmNq/lE0NiN9L77yh2JmVl51HwiWHbZtJSl2wnMrFbVfCKAVD3kbqRmVqucCHA3UjOrbU4EuBupmdU2J4KMu5GaWa1yIsgMG5a6kY4ZU+5IzMw6lhNBZqed0roErh4ys1rjRJDp0cOL2ptZbXIiyNHYjXTy5HJHYmbWcZwIcrgbqZnVIieCHO5Gama1yIkgz7Bh0NDgbqRmVjucCPI0zkbqbqRmVitKlggk9ZXUIGmipJckHd1Eme9Jej7bHpe0SaniKZa7kZpZrSnlFcFC4LiI2ADYBjhK0oZ5Zd4AdoyIjYHfAFeUMJ6iuBupmdWakiWCiJgWEc9kt+cCE4HeeWUej4j3s7tPAn1KFU9ruBupmdWSDmkjkNQf2AwY20yx7wNNfg+XNELSOEnjZs2aVYIIFzdsWPrpqwIzqwUlTwSSlgduB46JiDkFytSTEsGJTe2PiCsiYmBEDKyrqytdsJm1106bE4GZ1YKSJgJJ3UlJ4KaIuKNAmY2Bq4A9I+K9UsbTGkOHuhupmdWGUvYaEnA1MDEiLihQph9wB3BQRLxWqliWhLuRmlmt6FbCY28PHAS8IOm57LGfA/0AImIU8EtgVeDSlDdYGBEDSxhT0Rq7kd53HwweXO5ozMxKRxFR7hhaZeDAgTFu3LgOea2hQ1PPodcq6lrFzKz1JI0v9EXbI4ubMWwYvP66u5GaWefmRNAMz0ZqZrXAiaAZ7kZqZrXAiaAFjYvaz59f7kjMzErDiaAFQ4emsQTuRmpmnZUTQQs8G6mZdXZOBC3o0QPq650IzKzzciIowtChqRvppEnljsTMrP05ERTB3UjNrDNzIiiCu5GaWWfmRFCkxtlI3Y3UzDqbFhOBpO0lPSDpNUlTJL0haUpHBFcpzj0XevdevBtpQ0N63Mys2hUz++jVwE+B8cBnpQ2nMm25Jey7L3TvnqqHll463R89utyRmZm1XTFVQx9GxP0RMTMi3mvcSh5ZBamvX/Shf+ONi5JAfX154zIzaw/FJIIGSedJ2lbS5o1bySOrMPX18M1vwuzZMGSIk4CZdR7FVA1tnf3Mncc6gG+0fziVq6EBnnoKVloJbr4ZDjkEdtml3FGZmbVdi4kgImr+u29Dw6LqoHffTbf32gvuvttXBmZW/YrpNbSSpAskjcu230laqSOCqxRPP72oTWD4cPj616FrV3j00XJHZmbWdsW0EVwDzAX2zbY5wLWlDKrSnHDCom/+Elx4IcyZAx99VN64zMzaQzGJYEBE/CoipmTb6cBaLT1JUl9JDZImSnpJ0tFNlJGkiyRNkvR8tTRCb7EFHHoo/P73nn/IzKpfMYlgvqRBjXckbQ8UM752IXBcRGwAbAMcJWnDvDJDgXWybQRwWVFRV4Azz4SllkpXC2Zm1ayYRPAj4BJJb0p6C7gYGNnSkyJiWkQ8k92eC0wEeucV2xO4IZIngZ6SVmvVOyiT1VaDn/8c7rwzNSabmVWrFhNBRDwXEZsAGwMbRcRmETGhNS8iqT+wGTA2b1dv4O2c+1P5YrJA0ojGxupZs2a15qVL6qc/hTXWSD8/q8kx12bWGRRMBJIOzH4eK+lY4AjgiJz7RZG0PHA7cExEzMnf3cRT4gsPRFwREQMjYmBdXV2xL11yPXrAeefBhAlwzTXljsbMbMk0d0WwXPZzhSa25Ys5uKTupCRwU0Tc0USRqUDfnPt9gP8Uc+xKMXw4DBoEp5wCH35Y7mjMzFqv4ICyiLg8u/lgRPwrd1/WYNwsSSJNWDcxIi4oUOxu4MeSbiGNYP4wIqYVFXmFkFLvoS23hLPOgnPOKXdEZmatU0xj8R+LfCzf9sBBwDckPZdtwySNlNTY2HwfMAWYBFwJHFlM0JVmiy3SlBO//z1MnlzuaMzMWqfgFYGkbYHtgLq8NoEVga4tHTgiHqPpNoDcMgEcVVyole2ss+DWW+H44+GOpirBzMwqVHNXBEuR2gK6sXj7wBxgeOlDqy7uTmpm1UrpS3mBnVJX4C8RUTEf/AMHDoxx48aVO4wmzZ8PG2wAPXvC+PFpPiIzs0ogaXxEDGxqX7NtBBHxGbBKSaLqhHr0SMtXTpgA19bUbExmVs2KaSx+VtLdkg6StHfjVvLIqtR3vrOoO+mc/FETZmYVqJhEsArwHmkhmt2zbbdSBlXNGruTzpqV5iMyM6t0xSxMc1hHBNKZ5HYnHTECBgwod0RmZoUVszBNH0l3SpopaYak2yX16YjgqtmZZ0L37p6d1MwqXzFVQ9eSRgCvTpoQ7v+osYVplsTqq8PJJ6cxBQ8/XO5ozMwKKyYR1EXEtRGxMNuuAypn5rcKduyxaXbSY47x7KRmVrmKSQTvSjpQUtdsO5DUeGwtcHdSM6sGxSSCw0lrFU8HppFGFbsBuUjf+Q5sv727k5pZ5SomEfSNiD0ioi4ivhQR32bxqaOtGY3dSWfOTPMRmZlVmlLOPmqZgQNTd9ILL/TspGZWeUo2+6gtrk8f6NIldSe9/fb0WEMDPP20u5iaWXl59tEOsvPOKRE0didtaIB9900L2piZlVOzs48CSFojIt7KbncBlm9i7eEOU8mzj7bkb3+DXXeFXr3g889h9Giory93VGZWC5Z49tHM2ZJWlLQc8DLwqqTj2zXCGjFkSOpFNHMmrLoq7LBDuSMyMysuEWyYXQF8m7S0ZD/SEpTWSg0N8M9/wuDB8OqrsPfe0MIFmZlZyRWTCLpL6k5KBH+NiE8Bf3y1UmObwOjRqYpov/3g7rvhBz8od2RmVuuKSQSXA28CywGPSFqD1GBsrfD004u3Cfz5z6kB+eqr4brryhqamdW4FhuLm3yS1C0iFrZQ5hrSugUzI+JrTexfCbiRVNXUDTg/IlqciKGaG4vzLViQGo8bGuCee1IbgplZKTTXWNzcOIIDI+LGvDEEuS5o4XWvAy4Gbiiw/yjg5YjYXVIdqRH6pohY0MJxO42llkpjCnbcEYYPT91KBzb5azIzK53mqoaWy36uUGBrVkQ8AsxurgiwgiSRxivMBpq9yuiMVlwR7r8f6urS1YFHHptZR1uiqqGiDy71B+4pUDW0Ammdg/VJiWW/iLi3wHFGACMA+vXrt8Vbb71VspjL5dVXYbvtYJVV4F//gi99qdwRmVlnsqRVQxc1d9CI+H9tjGsw8BxpLeQBwAOSHm1qsFpEXAFcAamNoI2vW5HWWy+1E+y8M+y2W2o3WG65lp9nZtZWzVUNjc+2ZYDNgdezbVOgPZZZOQy4I5JJwBukq4Oate22cMstMH586mr66afljsjMakHBRBAR10fE9cA6QH1E/DEi/gjsTEoGbfXv7FhI+jKwHjClHY5b1fbYAy67DO67D0aO9IAzMyu9glVDOVYn1eE3Nvwunz3WLEk3AzsBvSRNBX4FdAeIiFHAb4DrJL0ACDgxIt5t7RvojEaMgHfegV//Os1aevrp5Y7IzDqzYhLBb4FnJTVk93cETmvpSRHx3Rb2/wf4VhGvX5NOOw2mTk3JYPXV4Yc/LHdEZtZZtZgIIuJaSfcDW2cPnRQR00sblkkwahRMnw5HHgmrrZaqjczM2lsxU0wQEdMj4q/Z5iTQQbp3T9NSbLEF7L8/PPFEuSMys86oqERg5bPccnDvvdC7N+y+expvYGbWnpwIqkBdXZqx9JNP0nQU06Yt2tfQAOeeW77YzKz6FZUIJA2SdFh2u07SmqUNy/INGADnnw8zZsDXvw5z5ni5SzNrHy02Fkv6FTCQ1M//WlIX0BuB7UsbmuUbORI+/BBOOgk22gjmzYNbb/Vyl2bWNsVcEewF7AHMg/91+2xx0jkrjRNPhL32gn//O1UVecCZmbVVMYlgQaSZ6QIgW7vYyqShAR59NA06++STNDfRL38JC2tu3lYzay/FJILRki4Hekr6AfAgcGVpw7Km5C53efnlaanLpZeG3/wmJYSpU8sdoZlVoxYTQUScD9wG3E5qJ/hlNueQdbD85S6HDUtrGey3X5qobtNNU1dTM7PWKOl6BKXQmZaqbE+vvpoSwoQJcOyxcPbZaQU0MzNofj2CFq8IJM2VNCdve1vSnZLWav9wbUmstx48+SQcdRRccAEMGgRTan4uVzMrRjFtBBcAxwO9gT7Az0htBLcA15QuNGutZZaBiy9O6yC/9hpstlmqSjIza04xiWBIRFweEXMjYk62WtiwiPgLsHKJ47MlsPfe8NxzsMEGqbpo5EiYP7/cUZlZpSomEXwuaV9JXbJt35x91dXAUEP690/dTE84IfUw2nprmDix3FGZWSUqJhF8DzgImAnMyG4fKKkH8OMSxmZt1L07nHNO6lk0fToMHJi6nz700OLlPF+RWW0rpvvolIjYPSJ6RURddntSRMyPiMc6IkhrmyFDUlXRVlulKSmGDVvUzdTzFZlZMXMNLQN8H/gqaSF7ACLi8BLGZe1s9dXhwQfhzDPT6md77gmHHQZ33bX42AQzqz3FVA39CfgKMBgYQ+o5NLeUQVlpdO2apqNoaIAePeCqq6Bv39SobGa1q5hEsHZEnArMi4jrgV2BjVp6kqRrJM2U9GIzZXaS9JyklySNKT5sa4vPP09TU2y9NTz7LKy5Jpx1lnsWmdWqYhLBp9nPDyR9DVgJ6F/E864DhhTaKakncCmwR0R8FfhOEce0NmpsE7j11jQA7frrU2I45RRYf324+WbPaGpWa4pJBFdIWhn4BXA38DJwTktPiohHgNnNFDkAuCMi/p2Vn1lELNZG+fMVHXxwWv1sxAhYdVU44ADYdlt4/PHyxmlmHafZRCCpCzAnIt6PiEciYq2I+FJEXN4Or70usLKkhyWNl3RwM3GMkDRO0rhZs2a1w0vXrhNO+GLDcH19Gmswbhxcdx28/TZsv30ajPbGG2UJ08w6ULOJICI+p3RjBboBW5DaHAYDp0pat0AcV0TEwIgYWFdXV6JwrEsXOOSQND3FaafBPfek6qITT0wro5lZ51RM1dADkn4mqa+kVRq3dnjtqcDfImJeRLwLPAJs0g7HtTZabjn41a9SQjjgADjvPFh7bbjsMi+AY9YZFZMIDgeOIn1Qj8+29pgH+q/A1yV1k7QssDXgSRAqSO/ecO21qcroq1+FI4+ETTaBww9Pjc65PDrZrHoVM7J4zSa2FqeflnQz8ASwnqSpkr4vaaSkkdlxJwJ/A54HngKuioiCXU2tfDbfPH3Q33UXLFiQksPgwXBNNvesRyebVbcWF6bJvq0fC/SLiBGS1gHWi4h7OiLAfF6YprwWLEhVRL/4BXz0UUoSb74Jt93m0clmlaxNC9MA1wILgO2y+1OBM9opNqsySy0FRx8Nb72V5i565pmUEMaOhU8+KXd0ZrYkikkEAyLiXLKBZRExH1BJo7KKN2FCWgHtyCPT/ZNPTj2MbrnFA9LMqk0xiWBBNuV0AEgaAPy3pFFZRWtsExg9Gi65JA1IW2kl6NYNvvtd2G47eOKJckdpZsUqJhGcRmrU7SvpJuCfwAmlDMoqW/7o5Pp6uPNOOOKI1ID81lspGey/f2o/MLPK1mJjMYCkVYFtSFVCT2b9/svCjcWV76OP0tiD885L8xgdcwz8/Oew4orljsysdrWpsVjS3cC3gIcj4p5yJgGrDssvD6efngak7bdfWiVt7bXTNBYekGZWeYqpGvod8HXgZUm3ShqeLVZj1qw+fdLspk8/nRqSR46ETTeFv/89DT7zoDSzylDMgLIxEXEksBZwBbAvaf1is6IMHAhjxsDtt6cupkOGpHEH++yzKBl4UJpZ+RRzRUDWa2gfYCSwJXB9KYOyzkeCvfeGl16C3/0uVRt98EFaP/knP1nUC8mD0sw6XjFtBH8hzQH0DeAS0riCn5Q6MOucll4ajj0WJk2CH/8Y/vtfuPji1HZw880pGbzrViizDlXsyOIBETEyIh4CtpV0SYnjsk6uVy/Yay9YeeVUVfTxx3DTTalxua4ONtsMjj8+jVGYN6/c0Zp1bsW0EfwN2EjSOZLeJE0v8UqpA7POrbFN4Lbb4P770wf+ssumAWpnnAE9e8JFF8HQoSlZ7LQT/OY3aaBaY88jNzibtY+CiUDSupJ+KWkicDFpjiFFRH1E/LHDIrROqalBaaNHpzEIp5ySPtDffz/1MDrmGJg7N62RsN12aUnNPfZIK6ntsw889FA6hhuczZZMwQFlkj4HHgW+HxGTssemFDMFdSl5QFnteu+99GH/4INpmzw5Pd6lS1pa8+WX4dZb3eBs1pQlHVC2DzAdaJB0paSd8WRzVkarrgrDh8OoUamx+Y034Kqr0mC1Rx9NXVOnTIHPPit3pGbVpWAiiIg7I2I/YH3gYeCnwJclXSbpWx0Un1lB/fvDWmvB7Nlw2GGpB9IRR6Q1Eh58sNzRmVWPYhqL50XETRGxG9AHeA44qdSBmbUkdxbUa65J7QkrrAAzZ8I3vwm77pqqi8yseUUNKGsUEbMj4vKI+EapAjIrVn6D8ze+AX/9axqgdt558NhjsPHGac2EmR4Lb1ZQUbOPVhI3FluxZs1Kk9+NGgXLLZdmQD36aFjGM2VZDWrrUpVL+qLXSJopqdkF6SVtKekzScNLFYvVprq6NGr5xRdhhx3gpJO8ippZU0qWCIDrgCHNFZDUFTgH+HsJ47Aat/768H//lxqQe/ZcfBU1D0ozK2EiiIhHgNktFPsJcDuezdQ6wM47w/jxi6+idv/9qUuqZ0G1WlbKK4JmSeoN7AWMKqLsCEnjJI2bNWtW6YOzTqtr19TV9LXX0kjlsWNhzpzUw+j44z0LqtWmsiUC4PfAiRHR4vCfiLgiIgZGxMC6urrSR2ad3vLLw2mnweuvw4EHwvz5cP75MGAArLtuuaMz61jlTAQDgVuyieyGA5dK+nYZ47Ea1Ls3HHxwmtjua19LVwhrrAE/+EFKEma1oGyJICLWjIj+EdEfuA04MiLuKlc8Vpsa2wRuvx1eeAFuvBG6d09LbK63Xtr37LPljtKstLqV6sCSbgZ2AnpJmgr8CugOEBEttguYdYT8QWnf+x6svnqa0XThQrj00jSR3eDBqfvpjjum1dbMOhMPKDNrxocfwmWXwYUXptHJ22wDJ58Mu+2WZj01qxZlGVBm1hmstFK6EnjzzXR1MH067LlnmrriT3+CTz/1WASrfk4EZkXo0QN+9KPUgHzjjal66OCDYZ114J13UluCxyJYtXIiMGuFbt1SO8KECWm0cu/eaUnNTz9N1UXHHeexCFZ9nAjMlkCXLumD/7HHYMyYNEr544/hggtSV9TZs9P6CGbVoGS9hsxqgZQmtPvsM3jyyTSv0dixadqKlVdO8xodfDBstZV7G1nl8hWBWRvljkV4/PG0QM5KK8Fmm6V5jbbZBjbcEM4+G6ZOLXe0Zl/kRGDWRvljEXbZBe68M409mDEjratcV5fWQ+jXL62eduONMG9eeeM2a+RxBGYdZPLk1OX0hhvgjTfSfEfDh8Mhh6TqpK22WryBuaEhJZkTTihfzNZ5eByBWQUYMCBNdDdpUmpgbqxOqq9PA9Z22y0lCnAXVOtYTgRmHaxLl9TAfPXVaYDaTTfBJpukXkcHHwwrrghDhsBee6UG5rlzyx2xdXZOBGZltOyycMABqYF56tTUrjB3blpj+cor09VCz56w0UZwxBFwxRVpDMPChYsfx6ObrS3cfdSsQrz2WlpB7dRT0/xGd94JSy+d2g/Gjk33r746lV12WdhiC9h669S20K/f4gPZGquWRo8u73uy6uBEYFYBcj+46+vT1nj/tNNSmYjU4PzUU4uSw0UXwYIFaf/KK6cqpe22g2eegVGjYKedyvWOrJq415BZBTj33NQw3NpeQwsWpKqisWNTgrj33jSqudGKK6YFdzbeOFUvNW49e7ZfDFYdmus15ERg1kk0XlUcfnhqSzj00JQoXnghbR98sKhsnz5fTA7Tp6d5lJqqXvK8SdWvuUTgqiGzTiD/Q3vIkEX3L7kkVStNnbooKTRuDzyQJsyDNKFenz4wdGjq1fTMM2lRHieBzs+JwKwTyB/dXF+f7j/9dLotQd++aRs2bNHzPv0UXn118eQwe3ZKEN27w913p/EP/fqV531Zx3DVkJn9T+OVxT77wHXXpUQhpcnzjj8+VSdZdfLIYjNrUW710qhRcP/9qVF5r71S19VNNknVRg8/nKqarPMoWSKQdI2kmZJeLLD/e5Kez7bHJW1SqljMrGVNVS/ddlvqSfT223DGGandoL4+jV+47bY0/bZVv5JVDUnaAfgIuCEivtbE/u2AiRHxvqShwGkRsXVLx3XVkFn5zJ+fJs07//w0Z9Laa8PPfpamxujRo9zRWXPKUjUUEY8As5vZ/3hEvJ/dfRLoU6pYzKx99OgBP/whvPJKuiJYeWUYORL694czz0yD3zzVRfWplDaC7wP3F9opaYSkcZLGzZo1qwPDMrOmdO2aGpTHjk0f9FtsAb/4BZxzDuy6K9xySyrnWVSrQ0l7DUnqD9zTVNVQTpl64FJgUES819IxXTVkVpmefx7OOw/+/Gf4/PM0SG3q1EVTbVt5VWyvIUkbA1cBexaTBMyscm28cVpPYcqUNBHeCy/A+++n+ZCeeqrc0VlzypYIJPUD7gAOiojXyhWHmbWvKVPSdtxxaZbUBx5IvYx22QUeeshdTytRKbuP3gw8Aawnaaqk70saKWlkVuSXwKrApZKek+T6HrMqlzsW4fzz4Z57YJllUgPzSy/BzjvDNtvAXXel6iOrDCWbYiIivtvC/iOAI0r1+mbW8Zoai3DrrenxN96A669PDcp77QUbbggnnwz775/mObLy8RQTZtahFi5MyeLss+HFF1PX0xNOgMMOS1cPVhoV21hsZrWnW7e0POeECWlSu698BY48MiWEc8+FX//aYxE6mhOBmZVFly6w++7w+OPpg37jjeHEE9MH/m67pXYE8FiEjuBEYGZlJaUlNf/xj9SWMHgwfPxxakfYZJOULH78Y1h1VZg3r9zRdk5uIzCzijNxYpr6esKEL+5bbTVYZ500z9Haay9+e/nlUxkvu/lFXqHMzKrK9Onwzjtw6qlw6aVw1llpSuxJk+D119PP++5L5XJ95SuLEsLpp6cJ8fbYI82e+oMfpEZq+yJfEZhZRclfdrO5tZPnzoXJkxclh9xEMW3a4mWXWSYliTXWSA3T/fsvfrtXr1RN1aizXVX4isDMqkZLy27mWmEF2HTTtOX76CM49li48kr45jfTuIU330zbY4/Bhx8uXn7ZZRdPDAsXpjUYfvvb1Mvp2WcXJaTOxlcEZtYpNV5J/OhHcNllX7yi+OADeOutRckh9/abb6Z5knJ16QKDBsGQIbD55mmrq+uod9N2viIws5qSX51UX//F6qWePdO2SYG1EefMScnhjDPS89ZbL82m+vOfLyrTt++ipNC4rb76ov3tUb3UEVVU7j5qZp1Oc9VLxVpxRXj33TRR3qmnwqxZcNVVMHt2euz889MVwiuvpAV5dt8devdOvZp23TU955NPYPjwVB6WbEzEllum5zQOsivFuApXDZmZNaG1jdYTJqQ1ncePTz9ffnnRxHoSrLJKqo768pdTw3VE2t/4s7nbn36axlYMGpS61jYVQ0tcNWRm1kqtbbQeNChtjT7+OK3J8MwzqcH62WdTg/Umm6T2Bin9LPb2k0/Co4+mK432XujHVwRmZiXUUqN1Rx3Dk86ZmZVBbnXSr3+dfubW93fUMVriRGBmViLt0WjdHsdoiauGzMxqgKuGzMysICcCM7Ma50RgZlbjnAjMzGqcE4GZWY2rul5DkmYBb5U7jhb0At4tdxBFcJztr1pidZztqxriXCMimpwvteoSQTWQNK5QN61K4jjbX7XE6jjbV7XEWYirhszMapwTgZlZjXMiKI0ryh1AkRxn+6uWWB1n+6qWOJvkNgIzsxrnKwIzsxrnRGBmVuOcCJaQpL6SGiRNlPSSpKObKLOTpA8lPZdtvyxTrG9KeiGL4QtTtyq5SNIkSc9L2rwMMa6Xc56ekzRH0jF5Zcp2PiVdI2mmpBdzHltF0gOSXs9+rlzguUMkvZqd35PKEOd5kl7Jfrd3SupZ4LnN/p10QJynSXon5/c7rMBzy30+/5IT45uSnivw3A47n20WEd6WYANWAzbPbq8AvAZsmFdmJ+CeCoj1TaBXM/uHAfcDArYBxpY53q7AdNIAmIo4n8AOwObAizmPnQuclN0+CTinwHuZDKwFLAVMyP876YA4vwV0y26f01ScxfyddECcpwE/K+Jvo6znM2//74Bflvt8tnXzFcESiohpEfFMdnsuMBHoXd6oltiewA2RPAn0lLRaGePZGZgcERUzgjwiHgFm5z28J3B9dvt64NtNPHUrYFJETImIBcAt2fM6LM6I+EdELMzuPgn0KdXrF6vA+SxG2c9nI0kC9gVuLtXrdxQngnYgqT+wGTC2id3bSpog6X5JX+3YyP4ngH9IGi9pRBP7ewNv59yfSnmT2v4U/ueqhPPZ6MsRMQ3SFwPgS02UqbRzezjp6q8pLf2ddIQfZ1VY1xSoaquk8/l1YEZEvF5gfyWcz6I4EbSRpOWB24FjImJO3u5nSNUbmwB/BO7q4PAabR8RmwNDgaMk7ZC3X008pyz9iiUtBewB3NrE7ko5n61RSef2FGAhcFOBIi39nZTaZcAAYFNgGqnaJV/FnE/guzR/NVDu81k0J4I2kNSdlARuiog78vdHxJyI+Ci7fR/QXVKvDg6TiPhP9nMmcCfp8jrXVKBvzv0+wH86JrovGAo8ExEz8ndUyvnMMaOxCi37ObOJMhVxbiUdAuwGfC+yCux8RfydlFREzIiIzyLic+DKAq9fKeezG7A38JdCZcp9PlvDiWAJZfWDVwMTI+KCAmW+kpVD0lak8/1ex0UJkpaTtELjbVLD4Yt5xe4GDs56D20DfNhY5VEGBb9lVcL5zHM3cEh2+xDgr02UeRpYR9Ka2dXO/tnzOoykIcCJwB4R8XGBMsX8nZRUXrvUXgVev+znM7ML8EpETG1qZyWcz1Ypd2t1tW7AINIl6fPAc9k2DBgJjMzK/Bh4idSz4UlguzLEuVb2+hOyWE7JHs+NU8AlpN4YLwADy3ROlyV9sK+U81hFnE9ScpoGfEr6Vvp9YFXgn8Dr2c9VsrKrA/flPHcYqVfZ5Mbz38FxTiLVqzf+nY7Kj7PQ30kHx/mn7O/vedKH+2qVeD6zx69r/LvMKVu289nWzVNMmJnVOFcNmZnVOCcCM7Ma50RgZlbjnAjMzGqcE4GZWY1zIjBrB5L6585QaVZNnAjMzGqcE4FZO5O0lqRnJW1Z7ljMiuFEYNaOJK1Hmn/qsIh4utzxmBWjW7kDMOtE6kjzDe0TES+VOxizYvmKwKz9fEia02f7cgdi1hq+IjBrPwtIq5T9XdJHEfHnMsdjVhQnArN2FBHzJO0GPCBpXkQ0NTW1WUXx7KNmZjXObQRmZjXOicDMrMY5EZiZ1TgnAjOzGudEYGZW45wIzMxqnBOBmVmN+/9CuNRIknxDkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8396624472573839\n",
      "0.8314606741573034\n",
      "0.8157276995305164\n"
     ]
    }
   ],
   "source": [
    "# finding optimal k\n",
    "from sklearn.cluster import KMeans \n",
    "from scipy.spatial.distance import cdist \n",
    "\n",
    "K = range(1, 20) \n",
    "meandistortions = [] \n",
    "for k in K: \n",
    "    kmeans = KMeans(n_clusters=k) \n",
    "    kmeans.fit(X_train) \n",
    "    meandistortions.append(sum(np.min(cdist(X_train, kmeans.cluster_centers_, 'euclidean'), axis=1)) / X_train.shape[0]) \n",
    "plt.plot(K, meandistortions, 'bx-') \n",
    "plt.xlabel('k') \n",
    "plt.ylabel('Average distortion') \n",
    "plt.title('Selecting k with the Elbow Method') \n",
    "plt.show() \n",
    "\n",
    "# knn model\n",
    "model = KNeighborsClassifier(n_neighbors=9)\n",
    "model.fit(X_train, y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "\n",
    "print(model.score(X_train, y_train))\n",
    "print(accuracy_score(y_pred,y_test))\n",
    "print(cross_val_score(model,X_train,y_train,cv=10,scoring='accuracy').mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Actual prediction on given test set for submission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((889, 8), (418, 8), (889,))"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = train_df[selected]\n",
    "y_train = train_df['Survived']\n",
    "X_test = test_df[selected]\n",
    "X_train.shape,X_test.shape,y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9111361079865017\n",
      "0.8301966292134833\n"
     ]
    }
   ],
   "source": [
    "model = RandomForestClassifier(n_estimators=1000)\n",
    "\n",
    "model.fit(X_train, y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "\n",
    "print(model.score(X_train, y_train))\n",
    "print(cross_val_score(model,X_train,y_train,cv=10,scoring='accuracy').mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "submission = pd.DataFrame({\n",
    "        \"PassengerId\": test_df['PassengerId'],\n",
    "        \"Survived\": y_pred\n",
    "    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "submission.set_index('PassengerId').to_csv(\"submission.csv\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}