Evaluation and Refinement-localGA.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Description\n",
    "\n",
    "This notebook is used to process and evalutate the initial universe of generated smiles, and then retain the network applying techniques and principles from both transfer learning and genetic algorithms to increasingly improve molecule generation for the specific task of binding with cornovirus protease."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First process initial generated smiles for PyRx analysis "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from rdkit import Chem, DataStructs\n",
    "from rdkit.Chem import Descriptors\n",
    "import random\n",
    "import numpy as np\n",
    "import rdkit.Chem.PropertyMol"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9703"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load gen0_smiles.smi as gen0_table\n",
    "# generation/gen0_smiles.smi\n",
    "gen0_table = pd.read_csv('./generations/gen0_smiles.smi',sep=',', header=None)\n",
    "gen0 = list(gen0_table[0])[0:10000]\n",
    "len(gen0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def validate_mols(list_of_smiles):\n",
    "    \"\"\"\n",
    "    :param list_of_smiles: python list of smiles\n",
    "    \"\"\"\n",
    "    valid_mols = []\n",
    "    for smi in list_of_smiles:\n",
    "        mol = Chem.MolFromSmiles(smi)\n",
    "        if mol is not None:\n",
    "            valid_mols.append(mol)\n",
    "    return valid_mols\n",
    "\n",
    "def convert_mols_to_smiles(list_of_mols):\n",
    "    valid_smiles = [Chem.MolToSmiles(mol) for mol in list_of_mols]\n",
    "    return valid_smiles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9703"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gen0_mols = validate_mols(gen0)\n",
    "len(gen0_mols)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Molecular Fingerprint\n",
    "Molecular fingerprints are a way of encoding the structure of a molecule. The most common type of fingerprint is a series of binary digits (bits) that represent the presence or absence of particular substructures in the molecule. Comparing fingerprints allows you to determine the similarity between two molecules, to find matches to a query substructure, etc.\n",
    "\n",
    "### Tanimoto Analysis\n",
    "One commonly used algorithm to calculate the similarity is the Tanimoto coefficient. In the Tanimoto Algorithm A and B are sets of fingerprint bits on in the fingerprints of molecule A and molecule B. AB is the set of common bits of fingerprints of both molecule A and B. The Tanimoto coefficient ranges from 0 when the fingerprints have no bits in common, to 1 when the fingerprints are identical."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def initialize_generation_from_mols(list_of_mols,desired_length): \n",
    "    '''\n",
    "    Intakes a list of smiles, randomly shuffles them, then adds first thirty,\n",
    "    then sets a max-similarity threshold between any new molecule and existing list\n",
    "    and iteratively increases the treshold until X components are picked to ensure diveristy\n",
    "    '''\n",
    "    assert desired_length >30\n",
    "    random.shuffle(list_of_mols)\n",
    "    random.shuffle(list_of_mols)\n",
    "    \n",
    "    # Prepare fingerprints for similarity calcs\n",
    "    mol_fingerprints = []\n",
    "    for mol in list_of_mols:\n",
    "        mol_fingerprints.append(Chem.RDKFingerprint(mol))\n",
    "    \n",
    "    selected_mols = list_of_mols[0:30]\n",
    "    selected_fingerprints = mol_fingerprints[0:30]\n",
    "    remaining_mols = list_of_mols[30:]\n",
    "    remaining_fingerprints = mol_fingerprints[30:]\n",
    "    \n",
    "    # Start with 0.05 as treshold\n",
    "    similarity_threshold = .05   \n",
    "    while len(selected_mols) < desired_length:\n",
    "        for fingerprint, mol in zip(remaining_fingerprints, remaining_mols):\n",
    "            max_similarity = np.max(DataStructs.BulkTanimotoSimilarity(fingerprint,selected_fingerprints))\n",
    "            if (max_similarity <= similarity_threshold) and (max_similarity < 1):\n",
    "                selected_fingerprints.append(fingerprint)\n",
    "                selected_mols.append(mol)\n",
    "        print(\"Completed loop with threshold at: \", similarity_threshold, \". Length is currently: \", len(selected_mols))\n",
    "        similarity_threshold += .05\n",
    "    return selected_mols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completed loop with threshold at:  0.05 . Length is currently:  30\n",
      "Completed loop with threshold at:  0.1 . Length is currently:  31\n",
      "Completed loop with threshold at:  0.15000000000000002 . Length is currently:  33\n",
      "Completed loop with threshold at:  0.2 . Length is currently:  38\n",
      "Completed loop with threshold at:  0.25 . Length is currently:  72\n",
      "Completed loop with threshold at:  0.3 . Length is currently:  202\n",
      "Completed loop with threshold at:  0.35 . Length is currently:  508\n",
      "Completed loop with threshold at:  0.39999999999999997 . Length is currently:  1122\n"
     ]
    }
   ],
   "source": [
    "# Select 1000 molecule based on tanimota similarity\n",
    "\n",
    "gen0_mols = initialize_generation_from_mols(gen0_mols,1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Questions: Where is this master_results_table.csv comes from (maybe from nothing)\n",
    "# generations/master_results_table.csv\n",
    "\n",
    "master_table = pd.read_csv('./generations/master_results_table.csv',sep=',')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'AAAB'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def iterate_alpha(alpha_code):\n",
    "    '''\n",
    "    Certainly not opimized and not strictly necessary, but in the PyRx GUI\n",
    "    molecule names would sort oddly when in any numeric order, so ordering\n",
    "    molcules by a four letter code. This function iterates the four letter code.\n",
    "    '''\n",
    "    numbers = []\n",
    "    for letter in alpha_code:\n",
    "        number = ord(letter)\n",
    "        numbers.append(number)\n",
    "    \n",
    "    if numbers[3]+1 > 90:\n",
    "        if numbers[2]+1 > 90:\n",
    "            if numbers[1]+1 > 90:\n",
    "                if numbers[0]+1 > 90:\n",
    "                    raise ValueError('Too long for alpha code')\n",
    "                else:\n",
    "                    numbers[3] = 65\n",
    "                    numbers[2] = 65\n",
    "                    numbers[1] = 65\n",
    "                    numbers[0] = numbers[0] + 1\n",
    "            else:\n",
    "                numbers[3] = 65\n",
    "                numbers[2] = 65\n",
    "                numbers[1] = numbers[1] + 1\n",
    "        else:\n",
    "            numbers[3] = 65\n",
    "            numbers[2] = numbers[2] + 1\n",
    "    else:\n",
    "        numbers[3] = numbers[3] + 1\n",
    "    \n",
    "\n",
    "    new_code = \"\"\n",
    "    for number in numbers:\n",
    "        new_code += chr(number)\n",
    "    return new_code\n",
    "iterate_alpha('AAAA')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAAC</td>\n",
       "      <td>0</td>\n",
       "      <td>CC(=O)Nc1ccc(S(=O)(=O)Nc2ccc(C)c(C)c2)cc1</td>\n",
       "      <td>training</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AAAD</td>\n",
       "      <td>0</td>\n",
       "      <td>CCOC(=O)C1=C(C(=O)OCC)C(c2cccc(Cl)c2)NC(=O)N1</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AAAE</td>\n",
       "      <td>0</td>\n",
       "      <td>NC(=O)c1ccc(NC(=O)C(CC(=O)O)NC(=O)c2cc(-c3cccc...</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AAAF</td>\n",
       "      <td>0</td>\n",
       "      <td>COC(=O)c1ccc2c(c1)N(C(=O)c1ccc(C)cc1)CC(C)(C)O2</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AAAG</td>\n",
       "      <td>0</td>\n",
       "      <td>Cc1ccc(C(=O)Nc2ccc(C(=O)N3CCOCC3)cc2)cc1</td>\n",
       "      <td>training</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2280</th>\n",
       "      <td>BBFT</td>\n",
       "      <td>0</td>\n",
       "      <td>CCCC1(CCc2ccccc2)CC(O)=C(C(CC)c2cccc(NS(=O)(=O...</td>\n",
       "      <td>manual</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2281</th>\n",
       "      <td>BBFU</td>\n",
       "      <td>0</td>\n",
       "      <td>O=C1Nc2ccc(Cl)cc2C(C#CC2CC2)(C(F)(F)F)O1</td>\n",
       "      <td>manual</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2282</th>\n",
       "      <td>BBFV</td>\n",
       "      <td>0</td>\n",
       "      <td>CC(C)(C)NC(=O)C1CN(Cc2cccnc2)CCN1CC(O)CC(Cc1cc...</td>\n",
       "      <td>manual</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2283</th>\n",
       "      <td>BBFW</td>\n",
       "      <td>0</td>\n",
       "      <td>CCOP(=O)(COc1ccc(CC(NC(=O)OC2COC3OCCC23)C(O)CN...</td>\n",
       "      <td>manual</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2284</th>\n",
       "      <td>BBFX</td>\n",
       "      <td>0</td>\n",
       "      <td>COC(=O)NC(C(=O)NCCCCC(CO)N(CC(C)C)S(=O)(=O)c1c...</td>\n",
       "      <td>manual</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2285 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  gen                                              smile     source  \\\n",
       "0     AAAC    0          CC(=O)Nc1ccc(S(=O)(=O)Nc2ccc(C)c(C)c2)cc1   training   \n",
       "1     AAAD    0      CCOC(=O)C1=C(C(=O)OCC)C(c2cccc(Cl)c2)NC(=O)N1  generated   \n",
       "2     AAAE    0  NC(=O)c1ccc(NC(=O)C(CC(=O)O)NC(=O)c2cc(-c3cccc...  generated   \n",
       "3     AAAF    0    COC(=O)c1ccc2c(c1)N(C(=O)c1ccc(C)cc1)CC(C)(C)O2  generated   \n",
       "4     AAAG    0           Cc1ccc(C(=O)Nc2ccc(C(=O)N3CCOCC3)cc2)cc1   training   \n",
       "...    ...  ...                                                ...        ...   \n",
       "2280  BBFT    0  CCCC1(CCc2ccccc2)CC(O)=C(C(CC)c2cccc(NS(=O)(=O...     manual   \n",
       "2281  BBFU    0           O=C1Nc2ccc(Cl)cc2C(C#CC2CC2)(C(F)(F)F)O1     manual   \n",
       "2282  BBFV    0  CC(C)(C)NC(=O)C1CN(Cc2cccnc2)CCN1CC(O)CC(Cc1cc...     manual   \n",
       "2283  BBFW    0  CCOP(=O)(COc1ccc(CC(NC(=O)OC2COC3OCCC23)C(O)CN...     manual   \n",
       "2284  BBFX    0  COC(=O)NC(C(=O)NCCCCC(CO)N(CC(C)C)S(=O)(=O)c1c...     manual   \n",
       "\n",
       "      score  \n",
       "0      99.9  \n",
       "1      99.9  \n",
       "2      99.9  \n",
       "3      99.9  \n",
       "4      99.9  \n",
       "...     ...  \n",
       "2280   99.9  \n",
       "2281   99.9  \n",
       "2282   99.9  \n",
       "2283   99.9  \n",
       "2284   99.9  \n",
       "\n",
       "[2285 rows x 5 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "master_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def append_to_tracking_table(master_table,mols_to_append, source, generation):\n",
    "    '''\n",
    "    This function is use to apend the mols to master table.\n",
    "    \n",
    "    :param master_table: pandas dataframe table\n",
    "    :param mols_to_append: molecule to append to the table\n",
    "    :param source: python string on how the molecule comes from ex.'generated', 'local_ga'\n",
    "    :generation: which generation of the genetic algorithm\n",
    "    \n",
    "    :return pandas.DataFrame of molecule, mols_to_export(type:sdkit)\n",
    "    '''\n",
    "    # Assign IDs for tracking to each mol, and assign a pandas table entry for each\n",
    "    mols_to_export = []\n",
    "    rows_list = []\n",
    "    \n",
    "    master_table_gen = master_table[master_table['gen'] == generation]\n",
    "    if master_table_gen.shape[0] == 0:\n",
    "        id_code = 'AAAA'\n",
    "    else:\n",
    "        master_table_gen_ids = master_table_gen.sort_values('id', ascending=True)\n",
    "        master_table_gen_max_id = master_table_gen_ids.tail(1)\n",
    "        key = master_table_gen_max_id['id'].keys()[0]\n",
    "        id_code = iterate_alpha(str(master_table_gen_max_id['id'][key])) # Call function iterate_alpha\n",
    "        \n",
    "    training_data = pd.read_csv('./datasets/all_smiles_clean.smi', header=None)\n",
    "    training_set = set(list(training_data[0]))\n",
    "    \n",
    "    for mol in mols_to_append:\n",
    "        pm = Chem.PropertyMol.PropertyMol(mol)\n",
    "        title = 'id' + str(id_code) + 'gen'+ str(generation)\n",
    "        # print(title)\n",
    "        \n",
    "        # Enables for tracking which molecule is which in PyRx GUI and PyRx results export\n",
    "        pm.SetProp('Title', title)\n",
    "        mols_to_export.append(pm)\n",
    "\n",
    "        #And track in pandas\n",
    "        mol_dict = {}\n",
    "        mol_dict['id'] = id_code\n",
    "        mol_dict['gen'] = generation\n",
    "        smile = Chem.MolToSmiles(mol)\n",
    "        assert type(smile) == type('string')\n",
    "        mol_dict['smile'] = smile\n",
    "\n",
    "        if (source!= 'hiv' and source != 'manual' and source != 'baseline') and (smile in training_set):\n",
    "            mol_dict['source'] = 'training'\n",
    "        else:\n",
    "            mol_dict['source'] = source\n",
    "        mol_dict['score'] = 99.9\n",
    "\n",
    "        rows_list.append(mol_dict)\n",
    "        id_code = iterate_alpha(id_code) # Call function iterate_alpha\n",
    "        \n",
    "    df = pd.DataFrame(rows_list)\n",
    "    return df, mols_to_export"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1119"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_mols_to_test = append_to_tracking_table(master_table,gen0_mols, 'generated', 0)\n",
    "mols_for_pd = new_mols_to_test[0]\n",
    "mols_for_export = new_mols_to_test[1]\n",
    "master_table = master_table.append(mols_for_pd)\n",
    "len(mols_for_export)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dropping ALL duplicate values \n",
    "master_table.drop_duplicates(subset =\"smile\", keep = False, inplace = True) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAAC</td>\n",
       "      <td>0</td>\n",
       "      <td>CC(=O)Nc1ccc(S(=O)(=O)Nc2ccc(C)c(C)c2)cc1</td>\n",
       "      <td>training</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AAAD</td>\n",
       "      <td>0</td>\n",
       "      <td>CCOC(=O)C1=C(C(=O)OCC)C(c2cccc(Cl)c2)NC(=O)N1</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AAAE</td>\n",
       "      <td>0</td>\n",
       "      <td>NC(=O)c1ccc(NC(=O)C(CC(=O)O)NC(=O)c2cc(-c3cccc...</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AAAF</td>\n",
       "      <td>0</td>\n",
       "      <td>COC(=O)c1ccc2c(c1)N(C(=O)c1ccc(C)cc1)CC(C)(C)O2</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AAAG</td>\n",
       "      <td>0</td>\n",
       "      <td>Cc1ccc(C(=O)Nc2ccc(C(=O)N3CCOCC3)cc2)cc1</td>\n",
       "      <td>training</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1113</th>\n",
       "      <td>BCWT</td>\n",
       "      <td>0</td>\n",
       "      <td>O=c1nc(N2CCOCC2)oc2cc3c(cc12)OCCO3</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1114</th>\n",
       "      <td>BCWU</td>\n",
       "      <td>0</td>\n",
       "      <td>Cc1ccc(-c2nc3ccccc3c3cc(C)ccc23)c(C)c1</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1115</th>\n",
       "      <td>BCWV</td>\n",
       "      <td>0</td>\n",
       "      <td>COc1ccc(C=Cc2nc(SCc3ccccc3)n[nH]2)cc1OC</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1117</th>\n",
       "      <td>BCWX</td>\n",
       "      <td>0</td>\n",
       "      <td>O=C(O)c1ccc2c(c1)[nH]c1ccc(NC(=O)c3ccccc3)cc12</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1118</th>\n",
       "      <td>BCWY</td>\n",
       "      <td>0</td>\n",
       "      <td>COCCOC(=O)N1CCC(S(=O)(=O)c2ccc(C)cc2)CC1</td>\n",
       "      <td>generated</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2626 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  gen                                              smile     source  \\\n",
       "0     AAAC    0          CC(=O)Nc1ccc(S(=O)(=O)Nc2ccc(C)c(C)c2)cc1   training   \n",
       "1     AAAD    0      CCOC(=O)C1=C(C(=O)OCC)C(c2cccc(Cl)c2)NC(=O)N1  generated   \n",
       "2     AAAE    0  NC(=O)c1ccc(NC(=O)C(CC(=O)O)NC(=O)c2cc(-c3cccc...  generated   \n",
       "3     AAAF    0    COC(=O)c1ccc2c(c1)N(C(=O)c1ccc(C)cc1)CC(C)(C)O2  generated   \n",
       "4     AAAG    0           Cc1ccc(C(=O)Nc2ccc(C(=O)N3CCOCC3)cc2)cc1   training   \n",
       "...    ...  ...                                                ...        ...   \n",
       "1113  BCWT    0                 O=c1nc(N2CCOCC2)oc2cc3c(cc12)OCCO3  generated   \n",
       "1114  BCWU    0             Cc1ccc(-c2nc3ccccc3c3cc(C)ccc23)c(C)c1  generated   \n",
       "1115  BCWV    0            COc1ccc(C=Cc2nc(SCc3ccccc3)n[nH]2)cc1OC  generated   \n",
       "1117  BCWX    0     O=C(O)c1ccc2c(c1)[nH]c1ccc(NC(=O)c3ccccc3)cc12  generated   \n",
       "1118  BCWY    0           COCCOC(=O)N1CCC(S(=O)(=O)c2ccc(C)cc2)CC1  generated   \n",
       "\n",
       "      score  \n",
       "0      99.9  \n",
       "1      99.9  \n",
       "2      99.9  \n",
       "3      99.9  \n",
       "4      99.9  \n",
       "...     ...  \n",
       "1113   99.9  \n",
       "1114   99.9  \n",
       "1115   99.9  \n",
       "1117   99.9  \n",
       "1118   99.9  \n",
       "\n",
       "[2626 rows x 5 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "master_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Append the results to master table.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the result to 'generations/master_results_table.csv'\n",
    "\n",
    "master_table = master_table.reset_index(drop=True)\n",
    "master_table.to_csv(r'./generations/master_results_table.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Manually Append HIV drug and few others drug to the master table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add HIV inhibitors manually into the table\n",
    "hiv_smiles = pd.read_csv('./datasets/hiv_inhibitors_cleaned.smi',sep=',', header=None)\n",
    "hiv_smiles = list(hiv_smiles[0])\n",
    "hiv_mols = validate_mols(hiv_smiles)\n",
    "\n",
    "master_table = pd.read_csv('./generations/master_results_table.csv',sep=',')\n",
    "new_mols_to_test = append_to_tracking_table(master_table,hiv_mols, 'hiv', 0)\n",
    "mols_for_pd = new_mols_to_test[0]\n",
    "mols_for_export = mols_for_export + new_mols_to_test[1]\n",
    "\n",
    "master_table = master_table.append(mols_for_pd)\n",
    "master_table = master_table.reset_index(drop=True)\n",
    "master_table.to_csv(r'./generations/master_results_table.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a few other smiles manually into the table (\"control group\" of training smiles)\n",
    "manual_smiles = pd.read_csv('./datasets/manual_testing_cleaned.smi',sep=',', header=None)\n",
    "manual_smiles = list(manual_smiles[0])\n",
    "manual_mols = validate_mols(hiv_smiles)\n",
    "\n",
    "master_table = pd.read_csv('./generations/master_results_table.csv',sep=',')\n",
    "new_mols_to_test = append_to_tracking_table(master_table,manual_mols, 'manual', 0)\n",
    "mols_for_pd = new_mols_to_test[0]\n",
    "mols_for_export = mols_for_export + new_mols_to_test[1]\n",
    "\n",
    "master_table = master_table.append(mols_for_pd)\n",
    "master_table = master_table.reset_index(drop=True)\n",
    "master_table.to_csv(r'./generations/master_results_table.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def write_gen_to_sdf(mols_for_export, generation, batch_size):\n",
    "    '''\n",
    "    Function to convert current generation of molecule to structure data files(SDF). To load in the PyRX to evaluate.\n",
    "    \n",
    "    :param mols_for_export: molecule to export (mol)\n",
    "    :param generation: generation of the Global GA\n",
    "    :param batch_size: maximum size of a single sdf files\n",
    "    \n",
    "    :return: ligands in sdf file format\n",
    "    \n",
    "    :output files: \n",
    "    If the incoming mol is more than batch size:\n",
    "        generations/gen(i)_batch_(i+1).sdf\n",
    "    Else:\n",
    "        generations/gen(i).sdf\n",
    "    '''\n",
    "    if len(mols_for_export) > batch_size:\n",
    "        batches = (len(mols_for_export) // 1000)+1\n",
    "        for i in range(0,batches):\n",
    "            batch_to_export = mols_for_export[i*batch_size:(i+1)*batch_size]\n",
    "            w = Chem.SDWriter('./generations/gen' +str(generation) + '_batch_' + str(i+1) + '.sdf')\n",
    "            for m in batch_to_export: w.write(m)\n",
    "    else:\n",
    "        w = Chem.SDWriter('./generations/gen' +str(generation) + '.sdf')\n",
    "        for m in mols_for_export:\n",
    "            w.write(m)\n",
    "    \n",
    "    # Noticed an issue where the very last line item of an sdf write is not written correctly until another arbitary write is made\n",
    "    w = Chem.SDWriter('./generations/junk/test.sdf')\n",
    "    w.write(m)\n",
    "    \n",
    "    return mols_for_export"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ok\n"
     ]
    }
   ],
   "source": [
    "write_gen_to_sdf(mols_for_export, 0, 2000)\n",
    "print('ok')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NOW GO TO PyRx: Analyze the SDF file and create a csv of binding score results (End of Generation 0)\n",
    "\n",
    "Saved the results in csv:\n",
    "\n",
    "generations/results/master_results_table_gen(i).csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Afterwards, process binding simulation results to 'evolve' the molecules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''This number must be MANUALLY iterated each generation. I did not write the entire process into a smooth function or loop but that would be the next steps.''' \n",
    "GLOBAL_GENERATION = 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>15619</th>\n",
       "      <td>ABAW</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1=CC(C(=O)Nc2nnnn2C2=CC=C(OC(=O)Nc3nnnnn3)C2...</td>\n",
       "      <td>generated</td>\n",
       "      <td>484.396</td>\n",
       "      <td>0.21518</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15620</th>\n",
       "      <td>ABAX</td>\n",
       "      <td>15</td>\n",
       "      <td>CCC(=O)NC1=CC=CC(C(=O)Nc2nnnn2N2CCN(C(=O)Nc3nn...</td>\n",
       "      <td>generated</td>\n",
       "      <td>532.485</td>\n",
       "      <td>-0.58700</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15621</th>\n",
       "      <td>ABAY</td>\n",
       "      <td>15</td>\n",
       "      <td>CCC(=O)CCC(=O)c1ccc(C#N)cc1</td>\n",
       "      <td>generated</td>\n",
       "      <td>215.252</td>\n",
       "      <td>2.50028</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15622</th>\n",
       "      <td>ABKA</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1=CC=CC(C(=O)Nc2nnnn2C2=CC=C(C(=O)Nc3nnnnn3)...</td>\n",
       "      <td>generated</td>\n",
       "      <td>545.523</td>\n",
       "      <td>1.99920</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15623</th>\n",
       "      <td>ABZZ</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCc2ccccc21</td>\n",
       "      <td>generated</td>\n",
       "      <td>132.206</td>\n",
       "      <td>2.73630</td>\n",
       "      <td>99.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  gen                                              smile  \\\n",
       "15619  ABAW   15  CC1=CC(C(=O)Nc2nnnn2C2=CC=C(OC(=O)Nc3nnnnn3)C2...   \n",
       "15620  ABAX   15  CCC(=O)NC1=CC=CC(C(=O)Nc2nnnn2N2CCN(C(=O)Nc3nn...   \n",
       "15621  ABAY   15                        CCC(=O)CCC(=O)c1ccc(C#N)cc1   \n",
       "15622  ABKA   15  CC1=CC=CC(C(=O)Nc2nnnn2C2=CC=C(C(=O)Nc3nnnnn3)...   \n",
       "15623  ABZZ   15                                     CC1CCc2ccccc21   \n",
       "\n",
       "          source   weight     logp  score  \n",
       "15619  generated  484.396  0.21518   99.9  \n",
       "15620  generated  532.485 -0.58700   99.9  \n",
       "15621  generated  215.252  2.50028   99.9  \n",
       "15622  generated  545.523  1.99920   99.9  \n",
       "15623  generated  132.206  2.73630   99.9  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the csv result analyzed in previous generation\n",
    "# './generations/master_results_table_gen(i-1).csv'\n",
    "master_table = pd.read_csv('./generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) + '.csv',sep=',')\n",
    "master_table.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(15624, 7)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "master_table.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "master_table.drop_duplicates(subset =\"smile\", keep = False, inplace = True) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(15624, 7)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "master_table.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RMSD values are calculated relative to the best mode and use only movable heavy atoms. Two variants of RMSD metrics are provided, rmsd/lb (RMSD lower bound) and rmsd/ub (RMSD upper bound), differing in how the atoms are matched in the distance calculation:\n",
    "\n",
    "* rmsd/ub matches each atom in one conformation with itself in the other conformation, ignoring any symmetry\n",
    "* rmsd' matches each atom in one conformation with the closest atom of the same element type in the other conformation (rmsd' can not be used directly, because it is not symmetric)\n",
    "* rmsd/lb is defined as follows: rmsd/lb(c1, c2) = max(rmsd'(c1, c2), rmsd'(c2, c1))\n",
    "\n",
    "So: if your ligand has no internal symmetry, rmsd/ub is the value to use. However, if you are docking e.g. benzene, this would give you different values depending on your ligand is rotates and which of the six equivalent CH point into your binding pocket, therefore rmsd/lb gives you a better ranking of docking solutions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ligand</th>\n",
       "      <th>Binding Affinity</th>\n",
       "      <th>rmsd/ub</th>\n",
       "      <th>rmsd/lb</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6LU7_idAAAAgen15_uff_E=47.54</td>\n",
       "      <td>-3.7</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6LU7_idAAAAgen15_uff_E=47.54</td>\n",
       "      <td>-3.7</td>\n",
       "      <td>0.017</td>\n",
       "      <td>1.396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6LU7_idAAAAgen15_uff_E=47.54</td>\n",
       "      <td>-3.7</td>\n",
       "      <td>0.046</td>\n",
       "      <td>1.978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6LU7_idAAAAgen15_uff_E=47.54</td>\n",
       "      <td>-3.7</td>\n",
       "      <td>0.092</td>\n",
       "      <td>1.979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6LU7_idAAAAgen15_uff_E=47.54</td>\n",
       "      <td>-3.7</td>\n",
       "      <td>18.560</td>\n",
       "      <td>19.109</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         Ligand  Binding Affinity  rmsd/ub   rmsd/lb\n",
       "0  6LU7_idAAAAgen15_uff_E=47.54              -3.7    0.000     0.000\n",
       "1  6LU7_idAAAAgen15_uff_E=47.54              -3.7    0.017     1.396\n",
       "2  6LU7_idAAAAgen15_uff_E=47.54              -3.7    0.046     1.978\n",
       "3  6LU7_idAAAAgen15_uff_E=47.54              -3.7    0.092     1.979\n",
       "4  6LU7_idAAAAgen15_uff_E=47.54              -3.7   18.560    19.109"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This is the result from PyRX\n",
    "# './generations/results/results_gen(i-1).csv'\n",
    "new_scores = pd.read_csv('./generations/results/results_gen' + str(GLOBAL_GENERATION-1) + '.csv',sep=',')\n",
    "new_scores.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Ligand', 'Binding Affinity', 'rmsd/ub', ' rmsd/lb'], dtype='object')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_scores.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "new_scores = new_scores.groupby(\"Ligand\").min()[\"Binding Affinity\"].reset_index()\n",
    "new_scores['id'] = new_scores['Ligand'].str.split(\"_\").str[1].str.split(\"gen\").str[0].str.split(\"id\").str[1]\n",
    "new_scores['gen'] = new_scores['Ligand'].str.split(\"_\").str[1].str.split(\"gen\").str[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "new_scores['Ligand'].str.split(\"_\").str[0].str.split(\"gen\").str[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAAA</td>\n",
       "      <td>15</td>\n",
       "      <td>-3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AAAA</td>\n",
       "      <td>15</td>\n",
       "      <td>-4.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AAAC</td>\n",
       "      <td>15</td>\n",
       "      <td>-10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AAAC</td>\n",
       "      <td>15</td>\n",
       "      <td>-3.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AABG</td>\n",
       "      <td>15</td>\n",
       "      <td>-2.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>460</th>\n",
       "      <td>ABZI</td>\n",
       "      <td>15</td>\n",
       "      <td>-13.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>461</th>\n",
       "      <td>ABZJ</td>\n",
       "      <td>15</td>\n",
       "      <td>-4.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>462</th>\n",
       "      <td>ABZN</td>\n",
       "      <td>15</td>\n",
       "      <td>-3.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>463</th>\n",
       "      <td>ABZU</td>\n",
       "      <td>15</td>\n",
       "      <td>-12.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>464</th>\n",
       "      <td>ABZY</td>\n",
       "      <td>15</td>\n",
       "      <td>-13.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>465 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       id gen  score\n",
       "0    AAAA  15   -3.7\n",
       "1    AAAA  15   -4.8\n",
       "2    AAAC  15  -10.0\n",
       "3    AAAC  15   -3.7\n",
       "4    AABG  15   -2.7\n",
       "..    ...  ..    ...\n",
       "460  ABZI  15  -13.9\n",
       "461  ABZJ  15   -4.6\n",
       "462  ABZN  15   -3.9\n",
       "463  ABZU  15  -12.2\n",
       "464  ABZY  15  -13.2\n",
       "\n",
       "[465 rows x 3 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_scores = new_scores.groupby(\"Ligand\").min()[\"Binding Affinity\"].reset_index()\n",
    "\n",
    "new_scores['id'] = new_scores['Ligand'].str.split(\"_\").str[1].str.split(\"gen\").str[0].str.split(\"id\").str[1]\n",
    "#new_scores['id'] = new_scores['Ligand'].str.split(\"_\").str[0].str.split(\"gen\").str[0].str.split(\"id\").str[1] # Temporary\n",
    "\n",
    "new_scores['gen'] = new_scores['Ligand'].str.split(\"_\").str[1].str.split(\"gen\").str[1]\n",
    "#new_scores['gen'] = new_scores['Ligand'].str.split(\"_\").str[0].str.split(\"gen\").str[1] # Temporary\n",
    "\n",
    "new_scores['score'] = new_scores[\"Binding Affinity\"]\n",
    "new_scores = new_scores[['id','gen','score']]\n",
    "new_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ABSQ</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1374.561</td>\n",
       "      <td>36.7417</td>\n",
       "      <td>-16.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ABEO</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1448.643</td>\n",
       "      <td>37.9623</td>\n",
       "      <td>-16.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABZJ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1312.235</td>\n",
       "      <td>32.3335</td>\n",
       "      <td>-16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ABOH</td>\n",
       "      <td>12</td>\n",
       "      <td>CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1286.367</td>\n",
       "      <td>34.1331</td>\n",
       "      <td>-15.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AAHJ</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1362.550</td>\n",
       "      <td>36.5756</td>\n",
       "      <td>-15.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>ABZD</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2C...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1488.708</td>\n",
       "      <td>39.0886</td>\n",
       "      <td>-15.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>ACIQ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1290.229</td>\n",
       "      <td>32.4477</td>\n",
       "      <td>-15.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>AALF</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1316.437</td>\n",
       "      <td>34.9573</td>\n",
       "      <td>-15.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ABJD</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1384.386</td>\n",
       "      <td>34.3280</td>\n",
       "      <td>-15.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ABKL</td>\n",
       "      <td>10</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1264.191</td>\n",
       "      <td>31.4891</td>\n",
       "      <td>-15.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>ABCP</td>\n",
       "      <td>14</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1344.491</td>\n",
       "      <td>35.7375</td>\n",
       "      <td>-15.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>AANI</td>\n",
       "      <td>12</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1292.245</td>\n",
       "      <td>32.2693</td>\n",
       "      <td>-15.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>ABIN</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1394.466</td>\n",
       "      <td>35.8504</td>\n",
       "      <td>-15.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>ABLJ</td>\n",
       "      <td>14</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1346.507</td>\n",
       "      <td>35.9615</td>\n",
       "      <td>-15.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>AAXU</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1272.255</td>\n",
       "      <td>32.3851</td>\n",
       "      <td>-15.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>AAWX</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC=CC=CC=...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1205.982</td>\n",
       "      <td>28.5268</td>\n",
       "      <td>-15.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>ABKO</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1382.540</td>\n",
       "      <td>36.4598</td>\n",
       "      <td>-15.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>AAMJ</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1302.410</td>\n",
       "      <td>34.5672</td>\n",
       "      <td>-15.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>ABDZ</td>\n",
       "      <td>9</td>\n",
       "      <td>CC1CCC(C(=O)Nc2nnnn2-c2nnnnn2)c2ccccc2-c2ccccc...</td>\n",
       "      <td>generated</td>\n",
       "      <td>871.021</td>\n",
       "      <td>11.9259</td>\n",
       "      <td>-15.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>AAZJ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1518.778</td>\n",
       "      <td>40.0928</td>\n",
       "      <td>-15.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>AACR</td>\n",
       "      <td>7</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1204.051</td>\n",
       "      <td>29.6248</td>\n",
       "      <td>-15.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>AAMB</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1314.251</td>\n",
       "      <td>33.2247</td>\n",
       "      <td>-15.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>AAIR</td>\n",
       "      <td>12</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1324.501</td>\n",
       "      <td>35.8533</td>\n",
       "      <td>-15.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>AAYZ</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1228.158</td>\n",
       "      <td>30.9908</td>\n",
       "      <td>-15.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>AAON</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1258.313</td>\n",
       "      <td>33.1729</td>\n",
       "      <td>-15.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>AAGY</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1260.329</td>\n",
       "      <td>33.5769</td>\n",
       "      <td>-15.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>AAKQ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1518.863</td>\n",
       "      <td>41.1370</td>\n",
       "      <td>-14.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>ABOK</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1328.363</td>\n",
       "      <td>33.9455</td>\n",
       "      <td>-14.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>AAXW</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1314.421</td>\n",
       "      <td>34.7333</td>\n",
       "      <td>-14.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>ABSF</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2C...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1498.788</td>\n",
       "      <td>40.2086</td>\n",
       "      <td>-14.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>ABBM</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1318.453</td>\n",
       "      <td>35.1813</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>ABTL</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1264.191</td>\n",
       "      <td>31.4891</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>AATP</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1582.950</td>\n",
       "      <td>42.5492</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>ABEE</td>\n",
       "      <td>7</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1242.021</td>\n",
       "      <td>26.9575</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>ABQM</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1194.226</td>\n",
       "      <td>31.8944</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>ABBI</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC=CC=CC=CC=C...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1111.825</td>\n",
       "      <td>26.3882</td>\n",
       "      <td>-14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>ACHL</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1356.332</td>\n",
       "      <td>33.7278</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>AAYT</td>\n",
       "      <td>14</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1210.099</td>\n",
       "      <td>30.1527</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>AAHF</td>\n",
       "      <td>14</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1234.121</td>\n",
       "      <td>30.6290</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>ABOC</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1198.088</td>\n",
       "      <td>30.1666</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>ABEG</td>\n",
       "      <td>11</td>\n",
       "      <td>C#CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1346.337</td>\n",
       "      <td>34.1241</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>ACCJ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1107.963</td>\n",
       "      <td>28.1519</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>AAWY</td>\n",
       "      <td>15</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1426.637</td>\n",
       "      <td>37.8541</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>ABJX</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1206.237</td>\n",
       "      <td>32.0605</td>\n",
       "      <td>-14.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>AAHU</td>\n",
       "      <td>10</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1286.197</td>\n",
       "      <td>32.5886</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>AAKG</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1256.297</td>\n",
       "      <td>33.1289</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>AAWC</td>\n",
       "      <td>12</td>\n",
       "      <td>CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1276.202</td>\n",
       "      <td>31.8352</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>AAPM</td>\n",
       "      <td>10</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1076.047</td>\n",
       "      <td>29.2795</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>AAUJ</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1278.218</td>\n",
       "      <td>32.7264</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>ABEB</td>\n",
       "      <td>11</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC=CC=CC=CC=CC=...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1213.961</td>\n",
       "      <td>28.7914</td>\n",
       "      <td>-14.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id  gen                                              smile     source  \\\n",
       "0   ABSQ   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "1   ABEO   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "2   ABZJ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "3   ABOH   12  CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "4   AAHJ   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "9   ABZD   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2C...  generated   \n",
       "7   ACIQ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCCCCCC...  generated   \n",
       "8   AALF   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "5   ABJD   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "6   ABKL   10  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "10  ABCP   14  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "11  AANI   12  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "12  ABIN   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "13  ABLJ   14  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...  generated   \n",
       "14  AAXU   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCC...  generated   \n",
       "15  AAWX   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC=CC=CC=...  generated   \n",
       "19  ABKO   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "18  AAMJ   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "16  ABDZ    9  CC1CCC(C(=O)Nc2nnnn2-c2nnnnn2)c2ccccc2-c2ccccc...  generated   \n",
       "17  AAZJ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "20  AACR    7  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "21  AAMB   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "22  AAIR   12  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "23  AAYZ   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "24  AAON   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "25  AAGY   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "26  AAKQ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "27  ABOK   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCC...  generated   \n",
       "28  AAXW   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "29  ABSF   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2C...  generated   \n",
       "34  ABBM   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "35  ABTL   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "33  AATP   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "31  ABEE    7  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "30  ABQM   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "32  ABBI   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC=CC=CC=CC=C...  generated   \n",
       "36  ACHL   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCCCCCCCCCCCCC...  generated   \n",
       "37  AAYT   14  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "38  AAHF   14  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "39  ABOC   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "40  ABEG   11  C#CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "41  ACCJ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "42  AAWY   15  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "43  ABJX   13  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "48  AAHU   10  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "52  AAKG   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "51  AAWC   12  CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "50  AAPM   10  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "49  AAUJ   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "46  ABEB   11  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC=CC=CC=CC=CC=...  generated   \n",
       "\n",
       "      weight     logp  score  \n",
       "0   1374.561  36.7417  -16.2  \n",
       "1   1448.643  37.9623  -16.2  \n",
       "2   1312.235  32.3335  -16.0  \n",
       "3   1286.367  34.1331  -15.8  \n",
       "4   1362.550  36.5756  -15.8  \n",
       "9   1488.708  39.0886  -15.6  \n",
       "7   1290.229  32.4477  -15.6  \n",
       "8   1316.437  34.9573  -15.6  \n",
       "5   1384.386  34.3280  -15.6  \n",
       "6   1264.191  31.4891  -15.6  \n",
       "10  1344.491  35.7375  -15.5  \n",
       "11  1292.245  32.2693  -15.5  \n",
       "12  1394.466  35.8504  -15.5  \n",
       "13  1346.507  35.9615  -15.4  \n",
       "14  1272.255  32.3851  -15.4  \n",
       "15  1205.982  28.5268  -15.4  \n",
       "19  1382.540  36.4598  -15.3  \n",
       "18  1302.410  34.5672  -15.3  \n",
       "16   871.021  11.9259  -15.3  \n",
       "17  1518.778  40.0928  -15.3  \n",
       "20  1204.051  29.6248  -15.2  \n",
       "21  1314.251  33.2247  -15.2  \n",
       "22  1324.501  35.8533  -15.1  \n",
       "23  1228.158  30.9908  -15.1  \n",
       "24  1258.313  33.1729  -15.1  \n",
       "25  1260.329  33.5769  -15.0  \n",
       "26  1518.863  41.1370  -14.9  \n",
       "27  1328.363  33.9455  -14.9  \n",
       "28  1314.421  34.7333  -14.9  \n",
       "29  1498.788  40.2086  -14.9  \n",
       "34  1318.453  35.1813  -14.8  \n",
       "35  1264.191  31.4891  -14.8  \n",
       "33  1582.950  42.5492  -14.8  \n",
       "31  1242.021  26.9575  -14.8  \n",
       "30  1194.226  31.8944  -14.8  \n",
       "32  1111.825  26.3882  -14.8  \n",
       "36  1356.332  33.7278  -14.7  \n",
       "37  1210.099  30.1527  -14.7  \n",
       "38  1234.121  30.6290  -14.7  \n",
       "39  1198.088  30.1666  -14.7  \n",
       "40  1346.337  34.1241  -14.7  \n",
       "41  1107.963  28.1519  -14.7  \n",
       "42  1426.637  37.8541  -14.7  \n",
       "43  1206.237  32.0605  -14.7  \n",
       "48  1286.197  32.5886  -14.6  \n",
       "52  1256.297  33.1289  -14.6  \n",
       "51  1276.202  31.8352  -14.6  \n",
       "50  1076.047  29.2795  -14.6  \n",
       "49  1278.218  32.7264  -14.6  \n",
       "46  1213.961  28.7914  -14.6  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Append Score and Weight columns\n",
    "\n",
    "new_scores.id = new_scores.id.astype(str)\n",
    "new_scores.gen = new_scores.gen.astype(int)\n",
    "master_table.id = master_table.id.astype(str)\n",
    "master_table.gen = master_table.gen.astype(int)\n",
    "\n",
    "# Merge result csv and mastertable iwth ['id' , 'gen']\n",
    "new_table = pd.merge(master_table, new_scores, on=['id','gen'], suffixes=('_old','_new'), how='left')\n",
    "\n",
    "# Column ['score'] is replace with new value\n",
    "new_table['score'] = np.where(new_table['score_new'].isnull(), new_table['score_old'], new_table['score_new'])\n",
    "\n",
    "# Remove column ['score_old', 'score_new']\n",
    "new_table = new_table.drop(['score_old','score_new'], axis=1)\n",
    "\n",
    "# Compute the weight of smile\n",
    "new_table['weight'] = new_table['smile'].apply(lambda x: Descriptors.MolWt(Chem.MolFromSmiles(x)))\n",
    "\n",
    "# Compute LogP of smile\n",
    "new_table['logp'] = new_table['smile'].apply(lambda x: Descriptors.MolLogP(Chem.MolFromSmiles(x)))\n",
    "\n",
    "# Sort the table\n",
    "new_table = new_table.sort_values('score', ascending=True)\n",
    "new_table.head(50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Saved the merge result to the master_table csv\n",
    "# './generations/master_results_table_gen(i-1).csv'\n",
    "\n",
    "new_table.to_csv(r'./generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) + '.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  a) Select best 35 from score (sorted)  - 55 to 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Select top 35 ranked by score for training data to refine the molecule generator RNN\n",
    "# Set function is used to save the list in ascending order and also remove duplicate (Almost similar to java hashmap)\n",
    "\n",
    "training_smiles = list(set(list(new_table.head(55)['smile'])))\n",
    "len(training_smiles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### b) Select 10 based on the nearest tanimoto distance (Similarity) - 45"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\user\\Anaconda3\\envs\\tfv2\\lib\\site-packages\\ipykernel_launcher.py:14: RuntimeWarning: divide by zero encountered in double_scalars\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "      <th>similarity_adj_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16565</th>\n",
       "      <td>AAAK</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC=CC</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>176.218</td>\n",
       "      <td>1.5994</td>\n",
       "      <td>-13.5</td>\n",
       "      <td>-36.300725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16585</th>\n",
       "      <td>AABB</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#C...</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>464.482</td>\n",
       "      <td>1.6402</td>\n",
       "      <td>-11.9</td>\n",
       "      <td>-31.998416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16580</th>\n",
       "      <td>AAAY</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC#CC#CC(=O)NC1=CCCC=C1</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>305.336</td>\n",
       "      <td>1.3802</td>\n",
       "      <td>-14.7</td>\n",
       "      <td>-28.825712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16573</th>\n",
       "      <td>AAAO</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC#CC#CC=CC</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>224.262</td>\n",
       "      <td>1.6062</td>\n",
       "      <td>-9.4</td>\n",
       "      <td>-25.276060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16569</th>\n",
       "      <td>AAAM</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC#CC=CC</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>200.240</td>\n",
       "      <td>1.6028</td>\n",
       "      <td>-8.9</td>\n",
       "      <td>-23.931589</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  gen                                              smile  \\\n",
       "16565  AAAK   17                               CC#CC#CC#CC#CC#CC=CC   \n",
       "16585  AABB   17  CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#CC#C...   \n",
       "16580  AAAY   17             CC#CC#CC#CC#CC#CC#CC#CC(=O)NC1=CCCC=C1   \n",
       "16573  AAAO   17                         CC#CC#CC#CC#CC#CC#CC#CC=CC   \n",
       "16569  AAAM   17                            CC#CC#CC#CC#CC#CC#CC=CC   \n",
       "\n",
       "                  source   weight    logp  score  similarity_adj_score  \n",
       "16565  genetic_algorithm  176.218  1.5994  -13.5            -36.300725  \n",
       "16585  genetic_algorithm  464.482  1.6402  -11.9            -31.998416  \n",
       "16580  genetic_algorithm  305.336  1.3802  -14.7            -28.825712  \n",
       "16573  genetic_algorithm  224.262  1.6062   -9.4            -25.276060  \n",
       "16569  genetic_algorithm  200.240  1.6028   -8.9            -23.931589  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_fingerprints = []\n",
    "\n",
    "# Computer fingerprint\n",
    "for smile in training_smiles:\n",
    "    training_fingerprints.append(Chem.RDKFingerprint(Chem.MolFromSmiles(smile)))\n",
    "\n",
    "# Calculate the similarity score based on weight and score\n",
    "def calc_similarity_score(row):\n",
    "    fingerprint = Chem.RDKFingerprint(Chem.MolFromSmiles(row['smile']))\n",
    "    \n",
    "    # Compute the tanimoto fingerprint\n",
    "    similarity = np.max(DataStructs.BulkTanimotoSimilarity(fingerprint,training_fingerprints))\n",
    "    \n",
    "    adj_factor = (1 / similarity) **.333\n",
    "    adj_score = row['score'] * adj_factor\n",
    "    return adj_score\n",
    "\n",
    "# Copy from master table\n",
    "similarity_adjusted = new_table.copy(deep=True)\n",
    "\n",
    "# All the weight which smaller than 900\n",
    "similarity_adjusted = similarity_adjusted[(similarity_adjusted['weight'] < 900) & (similarity_adjusted[\"logp\"] < 5)]\n",
    "\n",
    "# Create a new column named similarity_adj_score, and apply the fucntion\n",
    "similarity_adjusted['similarity_adj_score'] = similarity_adjusted.apply(calc_similarity_score, axis=1)\n",
    "\n",
    "# Sort the value based on similarity_adj_score Ascending\n",
    "similarity_adjusted = similarity_adjusted.sort_values('similarity_adj_score', ascending=True)\n",
    "similarity_adjusted.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "65"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Select top 5 ranked by similarity adjusted score for training data \n",
    "# to refine the molecule generator RNN (ensures diverity)\n",
    "\n",
    "training_smiles += list(similarity_adjusted.head(10)['smile'])\n",
    "len(training_smiles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### c) Select 10 based on the logP - 55"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "      <th>logp_adj_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2932</th>\n",
       "      <td>ABEH</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1CCCC1C(=O)Nc1nnnn1-c1ccc(C(=O)Nc2nnnn2N2CCN...</td>\n",
       "      <td>generated</td>\n",
       "      <td>640.589</td>\n",
       "      <td>0.00970</td>\n",
       "      <td>-11.1</td>\n",
       "      <td>-65.459243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3208</th>\n",
       "      <td>AAUZ</td>\n",
       "      <td>13</td>\n",
       "      <td>CC1C=CC=C(C(=O)N2CC3=CC=C(Oc4ccccc4)CC32)C(=O)...</td>\n",
       "      <td>generated</td>\n",
       "      <td>611.583</td>\n",
       "      <td>0.00960</td>\n",
       "      <td>-10.8</td>\n",
       "      <td>-63.910236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2509</th>\n",
       "      <td>ABPS</td>\n",
       "      <td>16</td>\n",
       "      <td>CC1CCCC(C(=O)Nc2nnnn2N2C=NN=NN2N2CCN(C(=O)Nc3n...</td>\n",
       "      <td>generated</td>\n",
       "      <td>562.519</td>\n",
       "      <td>0.02160</td>\n",
       "      <td>-11.5</td>\n",
       "      <td>-51.947868</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18029</th>\n",
       "      <td>ACGV</td>\n",
       "      <td>17</td>\n",
       "      <td>Cc1ccc(NC(=O)Nc2nnnn2N2C=NN=NN2N2C=NN=NN2C2=CC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>636.521</td>\n",
       "      <td>0.02262</td>\n",
       "      <td>-11.5</td>\n",
       "      <td>-51.155790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2890</th>\n",
       "      <td>AAUI</td>\n",
       "      <td>14</td>\n",
       "      <td>CC1C=CC(C(=O)Nc2nnnn2C2=CC=C(OC(=O)Nc3nnnnn3)C...</td>\n",
       "      <td>generated</td>\n",
       "      <td>517.422</td>\n",
       "      <td>0.03790</td>\n",
       "      <td>-11.1</td>\n",
       "      <td>-41.579485</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  gen                                              smile  \\\n",
       "2932   ABEH   13  CC1CCCC1C(=O)Nc1nnnn1-c1ccc(C(=O)Nc2nnnn2N2CCN...   \n",
       "3208   AAUZ   13  CC1C=CC=C(C(=O)N2CC3=CC=C(Oc4ccccc4)CC32)C(=O)...   \n",
       "2509   ABPS   16  CC1CCCC(C(=O)Nc2nnnn2N2C=NN=NN2N2CCN(C(=O)Nc3n...   \n",
       "18029  ACGV   17  Cc1ccc(NC(=O)Nc2nnnn2N2C=NN=NN2N2C=NN=NN2C2=CC...   \n",
       "2890   AAUI   14  CC1C=CC(C(=O)Nc2nnnn2C2=CC=C(OC(=O)Nc3nnnnn3)C...   \n",
       "\n",
       "          source   weight     logp  score  logp_adj_score  \n",
       "2932   generated  640.589  0.00970  -11.1      -65.459243  \n",
       "3208   generated  611.583  0.00960  -10.8      -63.910236  \n",
       "2509   generated  562.519  0.02160  -11.5      -51.947868  \n",
       "18029  generated  636.521  0.02262  -11.5      -51.155790  \n",
       "2890   generated  517.422  0.03790  -11.1      -41.579485  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def calc_logp_score(row):\n",
    "    \"\"\"\n",
    "    Calculate the logP score.\n",
    "    \n",
    "    :params row: pandas row\n",
    "    \"\"\"\n",
    "    adj_factor = (2 / row['logp']) ** .333\n",
    "    \n",
    "    if adj_factor < 1:\n",
    "        adj_score = 0\n",
    "    else:\n",
    "        adj_score = row['score'] * adj_factor\n",
    "    return adj_score\n",
    "\n",
    "# Copy from master table\n",
    "logp_adjusted = new_table.copy(deep=True)\n",
    "logp_adjusted.dropna()\n",
    "\n",
    "# Select all results that have logp <= 2\n",
    "logp_adjusted = logp_adjusted[(logp_adjusted[\"logp\"] <= 2) & (logp_adjusted[\"logp\"] > 0) & (logp_adjusted[\"logp\"] < 5) & (logp_adjusted[\"score\"] < -10)]\n",
    "\n",
    "# Apply function\n",
    "logp_adjusted['logp_adj_score'] = logp_adjusted.apply(calc_logp_score, axis=1)\n",
    "\n",
    "# Sort the logp_adj_score\n",
    "logp_adjusted = logp_adjusted.sort_values('logp_adj_score', ascending=True)\n",
    "logp_adjusted.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "130"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Select top 10 ranked by logp adjusted score for training data \n",
    "# to refine the molecule generator RNN (ensures diverity)\n",
    "\n",
    "training_smiles += list(logp_adjusted.head(65)['smile'])\n",
    "len(training_smiles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### d) Select 10 based on the weight of the score - 65"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "      <th>weight_adj_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16575</th>\n",
       "      <td>AAAQ</td>\n",
       "      <td>17</td>\n",
       "      <td>FC#Cc1ccccn1</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>121.114</td>\n",
       "      <td>1.3601</td>\n",
       "      <td>-16.6</td>\n",
       "      <td>-32.371821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16565</th>\n",
       "      <td>AAAK</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC=CC</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>176.218</td>\n",
       "      <td>1.5994</td>\n",
       "      <td>-13.5</td>\n",
       "      <td>-23.236022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16580</th>\n",
       "      <td>AAAY</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC#CC#CC#CC#CC(=O)NC1=CCCC=C1</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>305.336</td>\n",
       "      <td>1.3802</td>\n",
       "      <td>-14.7</td>\n",
       "      <td>-21.069247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16595</th>\n",
       "      <td>AABN</td>\n",
       "      <td>17</td>\n",
       "      <td>CC=CC1C#CC1</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>92.141</td>\n",
       "      <td>1.5858</td>\n",
       "      <td>-9.0</td>\n",
       "      <td>-19.223942</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16579</th>\n",
       "      <td>AAAT</td>\n",
       "      <td>17</td>\n",
       "      <td>CC#CC#CC#CC(CO)Nc1ccc2c(c1)N=CCN2</td>\n",
       "      <td>genetic_algorithm</td>\n",
       "      <td>277.327</td>\n",
       "      <td>1.6173</td>\n",
       "      <td>-11.7</td>\n",
       "      <td>-17.315388</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  gen                                   smile             source  \\\n",
       "16575  AAAQ   17                            FC#Cc1ccccn1  genetic_algorithm   \n",
       "16565  AAAK   17                    CC#CC#CC#CC#CC#CC=CC  genetic_algorithm   \n",
       "16580  AAAY   17  CC#CC#CC#CC#CC#CC#CC#CC(=O)NC1=CCCC=C1  genetic_algorithm   \n",
       "16595  AABN   17                             CC=CC1C#CC1  genetic_algorithm   \n",
       "16579  AAAT   17       CC#CC#CC#CC(CO)Nc1ccc2c(c1)N=CCN2  genetic_algorithm   \n",
       "\n",
       "        weight    logp  score  weight_adj_score  \n",
       "16575  121.114  1.3601  -16.6        -32.371821  \n",
       "16565  176.218  1.5994  -13.5        -23.236022  \n",
       "16580  305.336  1.3802  -14.7        -21.069247  \n",
       "16595   92.141  1.5858   -9.0        -19.223942  \n",
       "16579  277.327  1.6173  -11.7        -17.315388  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def calc_weight_score(row):\n",
    "    \"\"\"\n",
    "    Calculate the weight.\n",
    "    \n",
    "    :params row: pandas row\n",
    "    \"\"\"\n",
    "    adj_factor = (900 / row['weight']) ** .333\n",
    "    if adj_factor < 1:\n",
    "        adj_score = 0\n",
    "    else:\n",
    "        adj_score = row['score'] * adj_factor\n",
    "    return adj_score\n",
    "\n",
    "weight_adjusted = new_table.copy(deep=True)\n",
    "weight_adjusted['weight_adj_score'] = weight_adjusted.apply(calc_weight_score, axis=1)\n",
    "weight_adjusted = weight_adjusted.sort_values('weight_adj_score', ascending=True)\n",
    "weight_adjusted.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "140"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Select top X ranked by similarity adjusted score for training data \n",
    "# to refine the molecule generator RNN (ensures diverity)\n",
    "\n",
    "training_smiles += list(weight_adjusted.head(10)['smile'])\n",
    "len(training_smiles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### e) Select 5 randomly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-25-cff5e14f5db6>:2: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.config.list_physical_devices('GPU')` instead.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow\n",
    "tensorflow.test.is_gpu_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from copy import copy\n",
    "\n",
    "import keras\n",
    "\n",
    "from lstm_chem.utils.config import process_config\n",
    "from lstm_chem.model import LSTMChem\n",
    "from lstm_chem.generator import LSTMChemGenerator\n",
    "from lstm_chem.trainer import LSTMChemTrainer\n",
    "from lstm_chem.data_loader import DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading model architecture from experiments/2019-12-23/LSTM_Chem/model_arch.json ...\n",
      "Loading model checkpoint from experiments/2019-12-23/LSTM_Chem/checkpoints/LSTM_Chem-baseline-model-full.hdf5 ...\n",
      "Loaded the Model.\n"
     ]
    }
   ],
   "source": [
    "# Generate some with the base original model\n",
    "\n",
    "CONFIG_FILE = 'experiments/2019-12-23/LSTM_Chem/config.json'\n",
    "config = process_config(CONFIG_FILE)\n",
    "modeler = LSTMChem(config, session='generate')\n",
    "generator = LSTMChemGenerator(modeler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_number = 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00,  2.53it/s]\n"
     ]
    }
   ],
   "source": [
    "base_generated = generator.sample(num=sample_number)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "145"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "base_generated_mols = validate_mols(base_generated)\n",
    "base_generated_smiles = convert_mols_to_smiles(base_generated_mols)\n",
    "random.shuffle(base_generated_smiles)\n",
    "random.shuffle(base_generated_smiles)\n",
    "# Select X for training data to refine the molecule generator RNN (ensures diverity)\n",
    "training_smiles += base_generated_smiles[0:5]\n",
    "len(training_smiles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gen</th>\n",
       "      <th>smile</th>\n",
       "      <th>source</th>\n",
       "      <th>weight</th>\n",
       "      <th>logp</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ACLT</td>\n",
       "      <td>17</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1354.571</td>\n",
       "      <td>36.8575</td>\n",
       "      <td>-17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACDH</td>\n",
       "      <td>17</td>\n",
       "      <td>CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1470.734</td>\n",
       "      <td>39.2484</td>\n",
       "      <td>-16.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABKH</td>\n",
       "      <td>16</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1338.443</td>\n",
       "      <td>35.0655</td>\n",
       "      <td>-16.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACLV</td>\n",
       "      <td>17</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1298.463</td>\n",
       "      <td>35.2971</td>\n",
       "      <td>-16.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ABWF</td>\n",
       "      <td>16</td>\n",
       "      <td>CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CC...</td>\n",
       "      <td>generated</td>\n",
       "      <td>1342.390</td>\n",
       "      <td>34.3356</td>\n",
       "      <td>-16.7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     id  gen                                              smile     source  \\\n",
       "0  ACLT   17  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CC...  generated   \n",
       "1  ACDH   17  CC1C=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC...  generated   \n",
       "2  ABKH   16  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...  generated   \n",
       "3  ACLV   17  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CCC...  generated   \n",
       "4  ABWF   16  CC1CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC2CC...  generated   \n",
       "\n",
       "     weight     logp  score  \n",
       "0  1354.571  36.8575  -17.0  \n",
       "1  1470.734  39.2484  -16.9  \n",
       "2  1338.443  35.0655  -16.8  \n",
       "3  1298.463  35.2971  -16.8  \n",
       "4  1342.390  34.3356  -16.7  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the previous generation of csv again\n",
    "# './generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) + '.csv'\n",
    "\n",
    "master_table = pd.read_csv('./generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) + '.csv',sep=',')\n",
    "master_table.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the list of smiles to train on based on current generation\n",
    "with open('./generations/training/gen' + str(GLOBAL_GENERATION) + '_training.smi', 'w') as f:\n",
    "    for item in training_smiles:\n",
    "        f.write(\"%s\\n\" % item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GA Cross Over and Mutation (Local)\n",
    "### f) Select 10 from the cross-over and mutation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "import pathlib\n",
    "pathlib.Path().absolute()\n",
    "\n",
    "from genetic_algorithm import crossover as co\n",
    "from genetic_algorithm import mutate as mt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_file(file_name):\n",
    "    mol_list = []\n",
    "    with open(file_name, 'r') as file:\n",
    "        for smiles in file:\n",
    "            mol_list.append(Chem.MolFromSmiles(smiles))\n",
    "\n",
    "    return mol_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_initial_population(population_size, file_name):\n",
    "    \"\"\"\n",
    "    :param population_size: Size of the population\n",
    "    :param file_name: filename in smi\n",
    "    :return: population(list)\n",
    "    \"\"\"\n",
    "    mol_list = read_file(file_name)\n",
    "    population = []\n",
    "    for i in range(population_size):\n",
    "        # Select n random from the population\n",
    "        population.append(random.choice(mol_list))\n",
    "\n",
    "    return population"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def compute_logP_population(mol_list, mating_pool_size):\n",
    "    \"\"\"\n",
    "    Fitness Function. Select survivor of the population to the n number of mating pool based on the logP value. \n",
    "    \n",
    "    :param mol_list: python list of molecule in smiles\n",
    "    :param mating_pool_size: size of the mating pool\n",
    "    :return: population(python list)\n",
    "    \"\"\"\n",
    "    population = []\n",
    "    for mol in mol_list:\n",
    "        logP_val = Descriptors.MolLogP(mol)\n",
    "        mol_logP = [mol, logP_val]\n",
    "        \n",
    "        population.append(mol_logP)\n",
    "    \n",
    "    # Sort population by LogP\n",
    "    population.sort(key=lambda x:x[1])\n",
    "    \n",
    "    # find n number of mating pool size which around 1.35 - 1.8\n",
    "    best_population = [] # Ideal population\n",
    "    upper_population = []\n",
    "    lower_population = []\n",
    "    mean_logP = (1.35 + 1.8)/2\n",
    "    \n",
    "    for mol in population:\n",
    "        if 1.35 <= mol[1] <= 1.8:\n",
    "            best_population.append(mol)\n",
    "        elif mean_logP >= mol[1]:\n",
    "            upper_population.append(mol)\n",
    "        elif mean_logP <= mol[1]:\n",
    "            lower_population.append(mol)\n",
    "        \n",
    "    # Assign mating population to the best population if length is not 0\n",
    "    mating_population = []\n",
    "    if len(best_population) is not 0:\n",
    "        for mol in best_population:\n",
    "            mating_population.append(mol)\n",
    "        \n",
    "    if len(mating_population) < mating_pool_size:\n",
    "        # Remaining needed\n",
    "        remaining = mating_pool_size - len(mating_population)\n",
    "        \n",
    "        # While the list of the mating is less than remaining\n",
    "        i = 0 \n",
    "        while len(mating_population) < remaining:\n",
    "            # print(\"Total mating_population: \",len(mating_population))\n",
    "            # print(\"Total upper_population: \",len(upper_population))\n",
    "            # print(\"Total lower_population: \",len(lower_population))\n",
    "            \n",
    "            #if len(mating_population) is not 0:\n",
    "            try:\n",
    "                # print(\"upper_population\", upper_population[i])\n",
    "                mating_population.append(upper_population[i])\n",
    "            except:\n",
    "                pass\n",
    "                # print(\"Error When append upper population. Append lower population...\")\n",
    "            \n",
    "            #if len(mating_population) is not 0:\n",
    "            try:\n",
    "                # print(\"lower_population\", lower_population[i])\n",
    "                mating_population.append(lower_population[i])\n",
    "            except:\n",
    "                pass\n",
    "                # print(\"Error When append lower population. Append upper population...\")\n",
    "            \n",
    "            i+=1\n",
    "                \n",
    "    # Make Sure its is same with mating_pool_size\n",
    "    mating_population = mating_population[:mating_pool_size]\n",
    "    \n",
    "    # Sort mating_population by LogP\n",
    "    mating_population.sort(key=lambda x:x[1])\n",
    "            \n",
    "    return mating_population\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reproduce(mating_pool, population_size, mutation_rate):\n",
    "    '''\n",
    "    Function to crossover and mutate. \n",
    "    '''\n",
    "    new_population = []\n",
    "\n",
    "    while len(new_population) < population_size:\n",
    "        parent_A = random.choice(mating_pool)\n",
    "        parent_B = random.choice(mating_pool)\n",
    "        new_child = co.crossover(parent_A, parent_B)\n",
    "\n",
    "        if new_child is not None:            \n",
    "            mutated_child = mt.mutate(new_child, mutation_rate)\n",
    "            if mutated_child is not None:\n",
    "                # print(','.join([Chem.MolToSmiles(mutated_child),Chem.MolToSmiles(new_child),Chem.MolToSmiles(parent_A),Chem.MolToSmiles(parent_B)]))\n",
    "                new_population.append(mutated_child)\n",
    "\n",
    "    return new_population"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create initial population generation 0\n",
    "population_size = 300\n",
    "population = make_initial_population(population_size, './generations/training/gen' + str(GLOBAL_GENERATION) + '_training.smi')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 0: Minimum, Maximum: 1.3600999999999999, 1.751219999999999 \n",
      "\n",
      "Generation 1: Minimum, Maximum: 1.3601, 1.5959999999999999 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Cross-Over and Mutation\n",
    "mutation_rate = 0.05\n",
    "total_generation = 5000\n",
    "mating_pool = 50 # make sure this is smaller or equal to the population \n",
    "\n",
    "# generation 0\n",
    "gen_population = reproduce(population, population_size, mutation_rate)\n",
    "new_population_list = compute_logP_population(gen_population, mating_pool)\n",
    "new_population = [mol[0] for mol in new_population_list]\n",
    "Logp = [logP[1] for logP in new_population_list]\n",
    "minimum_LogP = min(Logp)\n",
    "maximum_LogP = max(Logp)\n",
    "\n",
    "# print(\"Generation 0: \\n{}\".format(Logp))\n",
    "print(\"Generation 0: Minimum, Maximum: {}, {} \\n\".format(minimum_LogP, maximum_LogP))\n",
    "\n",
    "# generation 1 and above\n",
    "for gen in range(1, total_generation):\n",
    "    gen_population = reproduce(new_population, population_size, mutation_rate)\n",
    "    new_population_list = compute_logP_population(gen_population, mating_pool)\n",
    "    \n",
    "    new_population = [mol[0] for mol in new_population_list]\n",
    "    Logp = [logP[1] for logP in new_population_list]\n",
    "    minimum_LogP = min(Logp)\n",
    "    maximum_LogP = max(Logp)\n",
    "    \n",
    "    # print(\"Generation {}: \\n{}\".format(gen, Logp))\n",
    "    print(\"Generation {}: Minimum, Maximum: {}, {} \\n\".format(gen, minimum_LogP, maximum_LogP))\n",
    "    \n",
    "    \n",
    "    if (min(Logp) >= 1.35 and max(Logp) <= 1.8):\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(new_population)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# new_population_smile = validate_mols(Chem.MolToSmiles(new_population))\n",
    "new_population_smile = []\n",
    "new_population_smile = [Chem.MolToSmiles(mol) for mol in new_population]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['FC#Cc1ccccn1',\n",
       " 'FC#Cc1ccccn1',\n",
       " 'FC#Cc1ccccn1',\n",
       " 'FC#CC#Cc1ccccn1',\n",
       " 'CC#CC=CC1=NN=NC1',\n",
       " 'CC=CC#CC1=NN=NC1',\n",
       " 'Cc1ccccn1',\n",
       " 'Cc1ccccn1',\n",
       " 'Cc1ccccn1',\n",
       " 'Cc1ccccn1']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_population_smile[0:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_population_smile = validate_mols(new_population_smile) # Validate\n",
    "new_population_smile = [Chem.MolToSmiles(mol) for mol in new_population] # Convert back to smile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "original_population_smile = [Chem.MolToSmiles(mol) for mol in population]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Final Population (Cross-Over + Heuristic Population)\n",
    "population_for_tl = original_population_smile + new_population_smile\n",
    "population_for_tl = list(dict.fromkeys(population_for_tl)) # Remove Duplicate in the list\n",
    "# population_for_tl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "152"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(population_for_tl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the list of smiles to train later in Transfer learning\n",
    "with open('./generations/training/gen' + str(GLOBAL_GENERATION) + '_training.smi', 'w') as f:\n",
    "    for item in population_for_tl:\n",
    "        f.write(\"%s\\n\" % item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Retrain the network to create molecules more like those selected above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lstm_chem.finetuner import LSTMChemFinetuner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "batch_size: 512\n",
      "checkpoint_dir: experiments\\2019-12-23\\LSTM_Chem\\checkpoints/\n",
      "checkpoint_mode: min\n",
      "checkpoint_monitor: val_loss\n",
      "checkpoint_save_best_only: false\n",
      "checkpoint_save_weights_only: true\n",
      "checkpoint_verbose: 1\n",
      "config_file: experiments/2019-12-23/LSTM_Chem/config.json\n",
      "data_filename: ./datasets/all_smiles_clean.smi\n",
      "data_length: 0\n",
      "exp_dir: experiments\\2019-12-23\\LSTM_Chem\n",
      "exp_name: LSTM_Chem\n",
      "finetune_batch_size: 1\n",
      "finetune_data_filename: ./generations/training/gen18_training.smi\n",
      "finetune_epochs: 5\n",
      "model_arch_filename: experiments/2019-12-23/LSTM_Chem/model_arch.json\n",
      "model_weight_filename: experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen17.hdf5\n",
      "num_epochs: 42\n",
      "optimizer: adam\n",
      "sampling_temp: 0.75\n",
      "seed: 71\n",
      "smiles_max_length: 128\n",
      "tensorboard_log_dir: experiments\\2019-12-23\\LSTM_Chem\\logs/\n",
      "tensorboard_write_graph: true\n",
      "train_smi_max_len: 128\n",
      "units: 256\n",
      "validation_split: 0.1\n",
      "verbose_training: true\n",
      "\n"
     ]
    }
   ],
   "source": [
    "config = process_config('experiments/2019-12-23/LSTM_Chem/config.json')\n",
    "config['model_weight_filename'] = 'experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen' + str(GLOBAL_GENERATION-1) + '.hdf5'\n",
    "config['finetune_data_filename'] = './generations/training/gen' + str(GLOBAL_GENERATION) + '_training.smi'\n",
    "print(config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading model architecture from experiments/2019-12-23/LSTM_Chem/model_arch.json ...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████████████████████████████████████████████████| 152/152 [00:00<00:00, 3306.97it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading model checkpoint from experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen17.hdf5 ...\n",
      "Loaded the Model.\n",
      "loading SMILES...\n",
      "done.\n",
      "tokenizing SMILES...\n",
      "done.\n",
      "WARNING:tensorflow:From C:\\Users\\user\\Documents\\UTeM Year 2 Sem 2\\BITI2513 - Introduction to Data Science\\Forkwel  Corona Virus Hackathon\\fch-drug-discovery\\lstm_chem\\finetuner.py:24: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use Model.fit, which supports generators.\n",
      "WARNING:tensorflow:sample_weight modes were coerced from\n",
      "  ...\n",
      "    to  \n",
      "  ['...']\n",
      "Train for 152 steps\n",
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "143/152 [===========================>..] - ETA: 1s - loss: 0.5179 - acc: 0.9002 - mae: 0.0059WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "152/152 [==============================] - 18s 121ms/step - loss: 0.5088 - acc: 0.9012 - mae: 0.0058\n",
      "Epoch 2/5\n",
      "WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "143/152 [===========================>..] - ETA: 1s - loss: 0.4034 - acc: 0.9098 - mae: 0.0054WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "152/152 [==============================] - 18s 117ms/step - loss: 0.4168 - acc: 0.9103 - mae: 0.0054\n",
      "Epoch 3/5\n",
      "WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "142/152 [===========================>..] - ETA: 1s - loss: 0.4015 - acc: 0.9154 - mae: 0.0052WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "152/152 [==============================] - 18s 117ms/step - loss: 0.3993 - acc: 0.9152 - mae: 0.0052\n",
      "Epoch 4/5\n",
      "WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "144/152 [===========================>..] - ETA: 0s - loss: 0.4022 - acc: 0.9141 - mae: 0.0052WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "152/152 [==============================] - 18s 121ms/step - loss: 0.3969 - acc: 0.9153 - mae: 0.0052\n",
      "Epoch 5/5\n",
      "WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "143/152 [===========================>..] - ETA: 1s - loss: 0.3793 - acc: 0.9120 - mae: 0.0050WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.\n",
      "152/152 [==============================] - 18s 120ms/step - loss: 0.3722 - acc: 0.9123 - mae: 0.0051\n"
     ]
    }
   ],
   "source": [
    "modeler = LSTMChem(config, session='finetune')\n",
    "finetune_dl = DataLoader(config, data_type='finetune')\n",
    "\n",
    "finetuner = LSTMChemFinetuner(modeler, finetune_dl)\n",
    "history = finetuner.finetune()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZhcdZ3v8fcnvWSj2ZJWIAl02GbSYaeJgYBGkBgBA/eCCMjmiLhxmVF0hFH0CurDxRkRHQSjg4OKICPjTAxgNGgyqCzpYFhCCFkMpgVME8lG1k6+94863VSaSqc6qVPr5/U8/eQsv1P17QPVnzqnvueUIgIzM7PeBpS6ADMzK08OCDMzy8kBYWZmOTkgzMwsJweEmZnlVF/qAgpl+PDh0dLSUuoyzMwqyty5c1+NiOZc66omIFpaWmhvby91GWZmFUXSizta51NMZmaWkwPCzMxyckCYmVlOVfMZhJnZrtiyZQsdHR1s3Lix1KWkatCgQYwcOZKGhoa8t3FAmFlN6+jooKmpiZaWFiSVupxURAQrV66ko6OD0aNH572dTzGZWU3buHEjw4YNq9pwAJDEsGHD+n2U5IAws5pXzeHQbVd+x5oPiFfXbeJLP5/P6g1bSl2KmVlZqfmA+Muajfz775fxrYcXlboUM6tBq1at4tvf/na/tzvjjDNYtWpVChW9oeYDYuwBe/H+tlH8+++XsbRzXanLMbMas6OA2Lp1a5/bPfjgg+y9995plQU4IAC4ZtLfMKihjq8+uKDUpZhZjbn22mtZsmQJxxxzDCeccALvfOc7ueiiizjyyCMBOOecczj++OMZO3YsU6dO7dmupaWFV199lWXLljFmzBg+/OEPM3bsWCZNmsSGDRsKUpvbXIHmpoFcdeqh3PTQ8zyyqJNTDst53yozq3Jf+vl8nntpTUEfs/WAPfnie8fucP1NN93Es88+y7x585g1axZnnnkmzz77bE876p133sm+++7Lhg0bOOGEEzj33HMZNmzYdo+xaNEi7rnnHr773e9y/vnnc//993PxxRfvdu0+gkh8cEILB+47hC9PX0DX1m2lLsfMatS4ceO2u1bhm9/8JkcffTTjx49n+fLlLFr05s9LR48ezTHHHAPA8ccfz7JlywpSi48gEgPr6/inM/6Wj/7oSe6Zs5xLxh9U6pLMrMj6eqdfLEOHDu2ZnjVrFjNnzuTRRx9lyJAhTJw4Mee1DAMHDuyZrqurK9gpJh9BZHn32P0Yf/C+fP2XC932amZF0dTUxNq1a3OuW716Nfvssw9Dhgzh+eef57HHHitqbQ6ILJK4/qxWVm3Y4rZXMyuKYcOGMWHCBI444gg+85nPbLdu8uTJdHV1cdRRR3H99dczfvz4otamiCjqE6alra0tCvWFQdfe/zQ/ndvBLz/5dg5u3qMgj2lm5WnBggWMGTOm1GUURa7fVdLciGjLNd5HEDm47dXMLOWAkDRZ0kJJiyVdm2P95ZI6Jc1Lfq7IWneZpEXJz2Vp1tlbd9vrzAUreGRRZzGf2sysbKQWEJLqgNuA9wCtwIWSWnMM/UlEHJP8fC/Zdl/gi8DbgHHAFyXtk1atuXS3vd44/Tm3vZpVuWo51d6XXfkd0zyCGAcsjoilEbEZuBc4O89t3w38KiL+GhGvAb8CJqdUZ07dba8v/GUd98xZXsynNrMiGjRoECtXrqzqkOj+PohBgwb1a7s0r4MYAWT/Ze0gc0TQ27mS3g68AHwyIpbvYNsRvTeUdCVwJcCBBx5YoLLfkN32OuXoA9hrcP7fxGRmlWHkyJF0dHTQ2Vndp5O7v1GuP9IMiFw3H+8d0T8H7omITZI+CtwFnJrntkTEVGAqZLqYdq/cN+tuez3rW7/lmw8v4vqzcp0hM7NK1tDQ0K9vWaslaZ5i6gBGZc2PBF7KHhARKyNiUzL7XeD4fLctlu67vd7lu72aWY1JMyDmAIdJGi2pEbgAmJY9QNL+WbNTgO6+0hnAJEn7JB9OT0qWlYTbXs2sFqUWEBHRBVxF5g/7AuC+iJgv6QZJU5JhV0uaL+kp4Grg8mTbvwI3kgmZOcANybKScNurmdUiX0mdp01dWzn96//DoIYBPHj1KdTX+RpDM6t8vpK6ADJtr2Pc9mpmNcMB0Q/vHvvWN+72ut53ezWz6uaA6Ifsu71+89e+26uZVTcHRD+NPWAvLjjBba9mVv0cELvgU6dn2l6/8oDbXs2sejkgdkF32+vDz6/gf15w26uZVScHxC7qvtvrlx/w3V7NrDo5IHbRdm2vT/yp1OWYmRWcA2I39LS9/uoFt72aWdVxQOwGSXzhrLFuezWzquSA2E2tB+zZ0/a6xG2vZlZFHBAF0N32+lW3vZpZFXFAFEBz00D+j9tezazKOCAK5HK3vZpZlXFAFIjbXs2s2jggCujdY9/KiQcPc9urmVUFB0QBZd/t9daH3fZqZpXNAVFg3W2vP3jUba9mVtkcECm4ZpLbXs2s8jkgUjB8D7e9mlnlSzUgJE2WtFDSYknX9jHuPEkhqS2Zb5B0l6RnJC2QdF2adaahu+31xuluezWzypRaQEiqA24D3gO0AhdKas0xrgm4Gng8a/H7gIERcSRwPPARSS1p1ZqG7rbXRSvc9mpmlSnNI4hxwOKIWBoRm4F7gbNzjLsRuBnYmLUsgKGS6oHBwGZgTYq1psJtr2ZWydIMiBHA8qz5jmRZD0nHAqMiYnqvbX8KvA68DPwJ+OeI+GvvJ5B0paR2Se2dneV3rt9tr2ZWydIMCOVYFj0rpQHALcA1OcaNA7YCBwCjgWskHfymB4uYGhFtEdHW3NxcmKoLzG2vZlap0gyIDmBU1vxI4KWs+SbgCGCWpGXAeGBa8kH1RcAvImJLRKwAfge0pVhrqtz2amaVKM2AmAMcJmm0pEbgAmBa98qIWB0RwyOiJSJagMeAKRHRTua00qnKGEomPJ5PsdZUue3VzCpRagEREV3AVcAMYAFwX0TMl3SDpCk72fw2YA/gWTJB8/2IeDqtWovh8gktHDTMba9mVjkUETsfVQHa2tqivb291GX0acb8V/jID+dy49ljueTEllKXY2aGpLkRkfMUvq+kLqJJrW57NbPK4YAoou6219VuezWzCuCAKLLWA/bk/Scc6LZXMyt7DogSuGbS4QxqqOMrbns1szLmgCiB7rbXXz+/gtluezWzMuWAKJHuttcvu+3VzMqUA6JEsu/2+mPf7dXMypADooTc9mpm5cwBUUKS+MJ7W1njtlczK0MOiBIbs/8bba+LV7jt1czKhwOiDFwz6XAGN9Tx1Qfd9mpm5cMBUQaG7zGQ/3Oa217NrLw4IMrEZSe57dXMyosDoky47dXMyo0DooxMan0rJx3itlczKw8OiDLSfbfXNRu28I2HXyh1OWZW4xwQZaa77fWHj77otlczKykHRBly26uZlQMHRBly26uZlQMHRJnqbnu90W2vZlYiqQaEpMmSFkpaLOnaPsadJykktWUtO0rSo5LmS3pG0qA0ay033W2vi932amYlklpASKoDbgPeA7QCF0pqzTGuCbgaeDxrWT3wI+CjETEWmAjUXN9ndtvrqvWbS12OmdWYNI8gxgGLI2JpRGwG7gXOzjHuRuBmYGPWsknA0xHxFEBErIyIrSnWWpay2159t1czK7Y0A2IEsDxrviNZ1kPSscCoiJjea9vDgZA0Q9KTkv4x1xNIulJSu6T2zs7q/DDXba9mVippBoRyLIueldIA4Bbgmhzj6oGTgQ8k//4vSae96cEipkZEW0S0NTc3F6bqMtTd9vqVB54rdSlmVkPSDIgOYFTW/Ejgpaz5JuAIYJakZcB4YFryQXUHMDsiXo2I9cCDwHEp1lrWuttef7Owk1kLV5S6HDOrEWkGxBzgMEmjJTUCFwDTuldGxOqIGB4RLRHRAjwGTImIdmAGcJSkIckH1u8Aavrtc8/dXh9Y4LZXMyuK1AIiIrqAq8j8sV8A3BcR8yXdIGnKTrZ9Dfg6mZCZBzwZEQ+kVWslGFhfx+eStte7H3fbq5mlTxGx81EVoK2tLdrb20tdRqoigg9873Gee3kNsz49kb2HNJa6JDOrcJLmRkRbrnW+krqCuO3VzIrJAVFhxuy/JxeMc9urmaXPAVGBPnW6217NLH0OiArktlczKwYHRIW6/KTRtLjt1cxS5ICoUI31A3ru9uq2VzNLgwOigp2e3O31lpm+26uZFZ4DooK57dXM0uSAqHBuezWztDggqoDbXs0sDQ6IKjB8j4Fcfdphbns1s4JyQFSJy05q6Wl73eK2VzMrAAdElchue/2x217NrAAcEFXk9Na3MuFQt72aWWE4IKqIJD5/Zqbt9Rsz3fZqZrvHAVFletpeH3uRxSvWlrocM6tgeQWEpL+XtKcy/k3Sk5ImpV2c7ZpPnX44Qxrq+MoDC0pdiplVsHyPIP4uItYAk4Bm4IPATalVZbvFba9mVgj5BoSSf88Avh8RT2UtszLktlcz2135BsRcSb8kExAzJDUB/qtTxtz2ama7K9+A+BBwLXBCRKwHGsicZuqTpMmSFkpaLOnaPsadJykktfVafqCkdZI+nWedlsVtr2a2O/INiBOBhRGxStLFwOeB1X1tIKkOuA14D9AKXCipNce4JuBq4PEcD3ML8FCeNVovbns1s92Rb0DcDqyXdDTwj8CLwA92ss04YHFELI2IzcC9wNk5xt0I3AxszF4o6RxgKTA/zxotB7e9mtmuyjcguiIiyPyBvzUibgWadrLNCGB51nxHsqyHpGOBURExvdfyocBngS/19QSSrpTULqm9s7Mzv9+kBl2TtL1+2W2vZtYP+QbEWknXAZcADySnjxp2sk2uLqfoWSkNIHMK6Zoc474E3BIRfX7BQURMjYi2iGhrbm7eSTm1a1jS9jprYSe/cdurmeUp34B4P7CJzPUQr5A5EvjaTrbpAEZlzY8EXsqabwKOAGZJWgaMB6YlH1S/Dbg5Wf4PwD9JuirPWi2H7rbXr7jt1czylFdAJKFwN7CXpLOAjRGxs88g5gCHSRotqRG4AJiW9ZirI2J4RLRERAvwGDAlItoj4pSs5d8AvhoR/9rv3856NNYP4HNntrJ4xTrufuzFUpdjZhUg31ttnA88AbwPOB94XNJ5fW0TEV3AVcAMYAFwX0TMl3SDpCm7V7btineNeUvS9rrIba9mtlPKfPa8k0HSU8DpEbEimW8GZkbE0SnXl7e2trZob28vdRll7/lX1nDGrY9w6Ykt/N8pY0tdjpmVmKS5EdGWa12+n0EM6A6HxMp+bGtl5G/325ML3fZqZnnI94/8LyTNkHS5pMuBB4AH0yvL0vQpt72aWR7y/ZD6M8BU4CjgaGBqRHw2zcIsPW57NbN81Oc7MCLuB+5PsRYrostOauHux1/ky9Of4+RDh9NQ5zOGZra9Pv8qSForaU2On7WS1hSrSCu87rbXJZ2vu+3VzHLqMyAioiki9szx0xQRexarSEuH217NrC8+r1DDJHH9Wa2s3ei7vZrZmzkgapzbXs1sRxwQlml7bXTbq5ltzwFhDNtjIH/vtlcz68UBYQBcemLmbq9fnv6c7/ZqZoADwhJuezWz3hwQ1uNdY97CyYcO55aZi3jtdbe9mtU6B4T1kMTnzxrD2o1buPVht72a1ToHhG3Hba9m1s0BYW/S3fZ643S3vZrVMgeEvUl32+vsF9z2albLHBCWk9tezcwBYTm57dXMHBC2Q257NattqQaEpMmSFkpaLOnaPsadJykktSXzp0uaK+mZ5N9T06zTcnPbq1ltSy0gJNUBtwHvAVqBCyW15hjXBFwNPJ61+FXgvRFxJHAZ8MO06rS+ue3VrHaleQQxDlgcEUsjYjNwL3B2jnE3AjcDG7sXRMQfIuKlZHY+MEjSwBRrtT647dWsNqUZECOA5VnzHcmyHpKOBUZFxPQ+Hudc4A8Rsan3CklXSmqX1N7Z2VmImi0Ht72a1aY0A0I5lkXPSmkAcAtwzQ4fQBoL/D/gI7nWR8TUiGiLiLbm5ubdLNf6cumJLYwePtRtr2Y1JM2A6ABGZc2PBF7Kmm8CjgBmSVoGjAemZX1QPRL4GXBpRCxJsU7LQ2P9AD53xhiWdL7Oj9z2alYT0gyIOcBhkkZLagQuAKZ1r4yI1RExPCJaIqIFeAyYEhHtkvYGHgCui4jfpVij9cNpSdvrN2YuYmnnulKXY2YpSy0gIqILuAqYASwA7ouI+ZJukDRlJ5tfBRwKXC9pXvLzlrRqtfx0t71u3LKVU/9lNmd96xG+PWsxL658vdSlmVkKFBE7H1UB2traor29vdRl1IQ/r9rAg0+/zAPPvMy85asAOGLEnpxx5P6ceeT+HDRsaIkrNLN8SZobEW051zkgbHd0vLaeh555xWFhVqEcEFYUDguzyuOAsKJzWJhVBgeElZTDwqx8OSCsbDgszMqLA8LKksPCrPQcEFb2HBZmpeGAsIrisDArHgeEVSyHhVm6HBBWFRwWZoXngLCqkyssxh6wJ2ce5bAw6w8HhFU1h4XZrnNAWM1wWJj1jwPCapLDwmznHBBW8xwWZrk5IMyy/HnVBh565mWmP+2wMHNAmO3AjsKiu3W2ZbjDwqqbA8IsDw4Lq0UOCLN+clhYrXBAmO0Gh4VVs5IFhKTJwK1AHfC9iLhpB+POA/4DOCEi2pNl1wEfArYCV0fEjL6eywFhxeCwsGpTkoCQVAe8AJwOdABzgAsj4rle45qAB4BG4KqIaJfUCtwDjAMOAGYCh0fE1h09nwPCis1hYdWgr4CoT/F5xwGLI2JpUsS9wNnAc73G3QjcDHw6a9nZwL0RsQn4o6TFyeM9mmK9Zv0yYu/BXHHKwVxxysHbhcXXZizkazMWbhcWBw0bgqRSl2zWL2kGxAhgedZ8B/C27AGSjgVGRcR0SZ/ute1jvbYd0fsJJF0JXAlw4IEHFqhss/7bWVgMEAxprGdwYx1DGusY0lif/Pvm+cGN9QztNT24sY6hA+sZ3JBZPnRg8lgNddTXDSj1r29VKs2AyPV2qed8lqQBwC3A5f3dtmdBxFRgKmROMe1SlWYF1jssZj73FzrXbmL95q2s39yV/JuZXruxixVrNvH65i42bN7K65u72LhlW7+er7F+QCZcGuoYMjATNIMb3giRoUkAdU8P3i6cdjw9qGGAj3pqXJoB0QGMypofCbyUNd8EHAHMSv4n3A+YJmlKHtuaVYQRew/mspNa+rXNtm3Bhi1be0IjV7Cs37w1EyibtrJ+S1fP9IYtybhNW1mxdiPrN22/Tde2/N9HSTCkITmKGVi3/dFLMj1kYH0mmBrfCKfuoMkE0vbT3UdQDT7qqQhpBsQc4DBJo4E/AxcAF3WvjIjVwPDueUmzgE8nH1JvAH4s6etkPqQ+DHgixVrNysaAAWLowHqGDiz8y3Nz17aeI5WekMkKotyhtH04rdvURefaTb3G7rB/JKfGugFZp9t6Hb0MrOekQ4Zx7nEjGdRQV/B9YPlLLSAiokvSVcAMMm2ud0bEfEk3AO0RMa2PbedLuo/MB9pdwCf66mAys/w01g+gsX4Aew1pKOjjbtsWbOza2nP0sj7rSGb95q7MEdGm7cNmw+YuXk9Can0y/eq6zby2Yh0PPP0yt/xqER86eTQXjz+QpkGFrdfy4wvlzKysRASPLlnJ7bOX8MiiV2kaVM8l4w/igxNG09w0sNTlVR1fSW1mFenpjlXcMXsJDz37Co11Azi/bRRXvv1gRu07pNSlVQ0HhJlVtCWd65g6eyn/+YcOtgW896j9+djEQ/mb/ZpKXVrFc0CYWVV4ZfVGvvfIUn78xJ9Yv3krp/3tW/j4Ow/h+IP2LXVpFcsBYWZVZdX6zdz1+xf599//kdfWb2Fcy7587J2HMPHwZl+70U8OCDOrSus3d3HvE8v57iNLeXn1Rsbsvycfm3gIZxyxn68wz5MDwsyq2uaubfz3vD9zx+wlLOl8nQP3HcJH3nGwr6XIgwPCzGrCtm3BL5/7C7fPWsxTHatpbhrIh04ezQfe5mspdsQBYWY1Jde1FJeemLmWYvgevpYimwPCzGpW72sp3n/CKD58iq+l6OaAMLOa52spcnNAmJklfC3F9hwQZma9+FqKDAeEmdkO1Pq1FA4IM7Od6H0txUHDhnDl26v/WgoHhJlZnmrtWgoHhJlZP3VfS/HtWUv47eLqvZbCAWFmthue7ljF7bOW8Iv51XcthQPCzKwAel9LMeXoA/joOw6p6GspHBBmZgX08uoN/Nsjf+y5luJdY97CxyZW5rUUfQVEqj1ckiZLWihpsaRrc6z/qKRnJM2T9FtJrcnyBkl3JesWSLouzTrNzPpj/70G8/mzWvndZ0/lk+86nLkvvsa5tz/K+d95lN8sXEG1vPFO7QhCUh3wAnA60AHMAS6MiOeyxuwZEWuS6SnAxyNisqSLgCkRcYGkIcBzwMSIWLaj5/MRhJmVSiVfS1GqI4hxwOKIWBoRm4F7gbOzB3SHQ2Io0J1WAQyVVA8MBjYD2WPNzMrGkMZ6/u7k0cz+zDv52nlHsblrK1ff8wdO+/ps7n78RTZu2VrqEndJmgExAlieNd+RLNuOpE9IWgLcDFydLP4p8DrwMvAn4J8j4q8p1mpmttsa6wfwvrZR/OqT7+COi49n78ENfO5nz3LKzb/hjtlLWLtxS6lL7Jc0AyLXzUzedD4rIm6LiEOAzwKfTxaPA7YCBwCjgWskHfymJ5CulNQuqb2zs7NwlZuZ7YYBA8TkI/bjvz4xgR9f8Tb+5q1N3PTQ85x006/52ozneXXdplKXmJc0A6IDGJU1PxJ4qY/x9wLnJNMXAb+IiC0RsQL4HfCmc2QRMTUi2iKirbm5uUBlm5kVhiROOnQ4P7ribUy7agInHzqcb89awoSbfs0X/vtZlv91falL7FOaATEHOEzSaEmNwAXAtOwBkg7Lmj0TWJRM/wk4VRlDgfHA8ynWamaWqqNG7s3tFx/PzE+9g3OOGcE9T/yJif88i0/+ZB4LX1lb6vJySvU6CElnAN8A6oA7I+Irkm4A2iNimqRbgXcBW4DXgKsiYr6kPYDvA61kTlV9PyK+1tdzuYvJzCpJ7mspDuX4g/Ypah2+UM7MrEy99vpmfvBo1vdSjN6Xj088hHcU6XspHBBmZmWu97UUrd3XUhy5P3UD0gsKB4SZWYXI9b0UH3n7Ifzv40ak8r0UDggzswqT63sprjh5NBcV+HspHBBmZhUqIvj9kpXcnnwvxZ6D6rn0xBYun9BSkO+lcECYmVWBp5av4o7Zhf1eCgeEmVkVWdK5ju/MXsLP/vBntgV88KQWPn9W6y49Vl8BUb9bVZqZWdEd0rwHN593NJ88/XC+98gfGbnP4FSexwFhZlah9t9rMNfv4pFDPsr7RuVmZlYyDggzM8vJAWFmZjk5IMzMLCcHhJmZ5eSAMDOznBwQZmaWkwPCzMxyqppbbUjqBF7cjYcYDrxaoHIKyXX1j+vqH9fVP9VY10ER0ZxrRdUExO6S1L6j+5GUkuvqH9fVP66rf2qtLp9iMjOznBwQZmaWkwPiDVNLXcAOuK7+cV3947r6p6bq8mcQZmaWk48gzMwsJweEmZnlVFMBIWmypIWSFku6Nsf6gZJ+kqx/XFJLmdR1uaROSfOSnyuKVNedklZIenYH6yXpm0ndT0s6rkzqmihpddb++kKR6hol6TeSFkiaL+nvc4wp+j7Ls66i7zNJgyQ9IemppK4v5RhT9NdknnWV6jVZJ+kPkqbnWFf4fRURNfED1AFLgIOBRuApoLXXmI8DdyTTFwA/KZO6Lgf+tQT77O3AccCzO1h/BvAQIGA88HiZ1DURmF6C/bU/cFwy3QS8kOO/ZdH3WZ51FX2fJftgj2S6AXgcGN9rTClek/nUVarX5KeAH+f6b5XGvqqlI4hxwOKIWBoRm4F7gbN7jTkbuCuZ/ilwmiSVQV0lERH/A/y1jyFnAz+IjMeAvSXtXwZ1lUREvBwRTybTa4EFwIhew4q+z/Ksq+iSfbAumW1Ifnp3zRT9NZlnXUUnaSRwJvC9HQwp+L6qpYAYASzPmu/gzS+SnjER0QWsBoaVQV0A5yanJH4qaVTKNeUr39pL4cTkFMFDksYW+8mTw/tjybz7zFbSfdZHXVCCfZacMpkHrAB+FRE73F9FfE3mUxcU/zX5DeAfgW07WF/wfVVLAZErSXu/K8hnTKHl85w/B1oi4ihgJm+8Syi1UuyvfDxJ5v4yRwPfAv6rmE8uaQ/gfuAfImJN79U5NinKPttJXSXZZxGxNSKOAUYC4yQd0WtISfZXHnUV9TUp6SxgRUTM7WtYjmW7ta9qKSA6gOyUHwm8tKMxkuqBvUj/VMZO64qIlRGxKZn9LnB8yjXlK599WnQRsab7FEFEPAg0SBpejOeW1EDmj/DdEfGfOYaUZJ/trK5S7rPkOVcBs4DJvVaV4jW507pK8JqcAEyRtIzMaehTJf2o15iC76taCog5wGGSRktqJPMhzrReY6YBlyXT5wG/juQTn1LW1esc9RQy55DLwTTg0qQzZzywOiJeLnVRkvbrPvcqaRyZ/89XFuF5BfwbsCAivr6DYUXfZ/nUVYp9JqlZ0t7J9GDgXcDzvYYV/TWZT13Ffk1GxHURMTIiWsj8jfh1RFzca1jB91X97mxcSSKiS9JVwAwynUN3RsR8STcA7RExjcyL6IeSFpNJ3gvKpK6rJU0BupK6Lk+7LgBJ95DpbhkuqQP4IpkP7IiIO4AHyXTlLAbWAx8sk7rOAz4mqQvYAFxQhKCHzLu8S4BnkvPXAP8EHJhVWyn2WT51lWKf7Q/cJamOTCDdFxHTS/2azLOukrwme0t7X/lWG2ZmllMtnWIyM7N+cECYmVlODggzM8vJAWFmZjk5IMzMLCcHhFkZUOZuqm+6Q6dZKTkgzMwsJweEWT9Iujj5roB5kr6T3NRtnaR/kfSkpIclNSdjj5H0WHJDt59J2idZfqikmcmN8Z6UdEjy8HskN357XtLdRbiTsFmfHBBmeZI0Bng/MCG5kdtW4APAUODJiDgOmE3mym6AHwCfTW7o9kzW8ruB25Ib450EdN9q41jgH4BWMt8PMiH1X2v1WYMAAAEZSURBVMqsDzVzqw2zAjiNzE3Z5iRv7geTuR30NuAnyZgfAf8paS9g74iYnSy/C/gPSU3AiIj4GUBEbARIHu+JiOhI5ucBLcBv0/+1zHJzQJjlT8BdEXHddgul63uN6+v+NX2dNtqUNb0Vvz6txHyKySx/DwPnSXoLgKR9JR1E5nV0XjLmIuC3EbEaeE3SKcnyS4DZyfcwdEg6J3mMgZKGFPW3MMuT36GY5SkinpP0eeCXkgYAW4BPAK8DYyXNJfMtXu9PNrkMuCMJgKW8cefWS4DvJHfi3AK8r4i/hlnefDdXs90kaV1E7FHqOswKzaeYzMwsJx9BmJlZTj6CMDOznBwQZmaWkwPCzMxyckCYmVlODggzM8vp/wMkDj7HijdQxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "\n",
    "pyplot.plot(history.history['loss'])\n",
    "pyplot.ylabel('loss')\n",
    "pyplot.xlabel('epoch')\n",
    "pyplot.legend(['train', 'validation'], loc='upper right')\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "finetuner.model.save_weights('experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen' + str(GLOBAL_GENERATION) + '.hdf5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading model architecture from experiments/2019-12-23/LSTM_Chem/model_arch.json ...\n",
      "Loading model checkpoint from experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen18.hdf5 ...\n",
      "Loaded the Model.\n",
      "batch_size: 512\n",
      "checkpoint_dir: experiments\\2019-12-23\\LSTM_Chem\\checkpoints/\n",
      "checkpoint_mode: min\n",
      "checkpoint_monitor: val_loss\n",
      "checkpoint_save_best_only: false\n",
      "checkpoint_save_weights_only: true\n",
      "checkpoint_verbose: 1\n",
      "config_file: experiments/2019-12-23/LSTM_Chem/config.json\n",
      "data_filename: ./datasets/all_smiles_clean.smi\n",
      "data_length: 0\n",
      "exp_dir: experiments\\2019-12-23\\LSTM_Chem\n",
      "exp_name: LSTM_Chem\n",
      "finetune_batch_size: 1\n",
      "finetune_data_filename: ./generations/training/gen18_training.smi\n",
      "finetune_epochs: 5\n",
      "model_arch_filename: experiments/2019-12-23/LSTM_Chem/model_arch.json\n",
      "model_weight_filename: experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen18.hdf5\n",
      "num_epochs: 42\n",
      "optimizer: adam\n",
      "sampling_temp: 0.75\n",
      "seed: 71\n",
      "smiles_max_length: 128\n",
      "tensorboard_log_dir: experiments\\2019-12-23\\LSTM_Chem\\logs/\n",
      "tensorboard_write_graph: true\n",
      "train_smi_max_len: 128\n",
      "units: 256\n",
      "validation_split: 0.1\n",
      "verbose_training: true\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# used the fine tuned weight to generate 5k of data\n",
    "config['model_weight_filename'] = 'experiments/2019-12-23/LSTM_Chem/checkpoints/finetuned_gen' + str(GLOBAL_GENERATION) + '.hdf5'\n",
    "modeler = LSTMChem(config, session='generate')\n",
    "generator = LSTMChemGenerator(modeler)\n",
    "print(config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████| 5000/5000 [1:27:10<00:00,  1.05s/it]\n"
     ]
    }
   ],
   "source": [
    "sample_number = 5000\n",
    "sampled_smiles = generator.sample(num=sample_number)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validity:  87.44%\n",
      "Uniqueness:  61.55%\n",
      "Originality:  100.00%\n"
     ]
    }
   ],
   "source": [
    "valid_mols = []\n",
    "for smi in sampled_smiles:\n",
    "    mol = Chem.MolFromSmiles(smi)\n",
    "    if mol is not None:\n",
    "        valid_mols.append(mol)\n",
    "# low validity\n",
    "print('Validity: ', f'{len(valid_mols) / sample_number:.2%}')\n",
    "valid_smiles = [Chem.MolToSmiles(mol) for mol in valid_mols]\n",
    "\n",
    "# high uniqueness\n",
    "print('Uniqueness: ', f'{len(set(valid_smiles)) / len(valid_smiles):.2%}')\n",
    "\n",
    "# Of valid smiles generated, how many are truly original vs ocurring in the training data\n",
    "import pandas as pd\n",
    "training_data = pd.read_csv('./datasets/all_smiles_clean.smi', header=None)\n",
    "training_set = set(list(training_data[0]))\n",
    "original = []\n",
    "for smile in list(set(valid_smiles)):\n",
    "    if not smile in training_set:\n",
    "        original.append(smile)\n",
    "print('Originality: ', f'{len(set(original)) / len(set(valid_smiles)):.2%}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2691"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valid_smiles = list(set(valid_smiles))\n",
    "len(valid_smiles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "mols_for_next_generation = validate_mols(valid_smiles)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "# take the valid smiles from above and run them through process \n",
    "# to add to tracking table and to generate next PyRx testing data\n",
    "mols_for_next_generation = validate_mols(valid_smiles)\n",
    "\n",
    "# Get the master table from previous generations and append it \n",
    "master_table = pd.read_csv('./generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) +'.csv',sep=',')\n",
    "new_mols_to_test = append_to_tracking_table(master_table,mols_for_next_generation, 'generated', GLOBAL_GENERATION)\n",
    "mols_for_pd = new_mols_to_test[0]\n",
    "mols_for_export = new_mols_to_test[1]\n",
    "\n",
    "# Save the whole details in new table with current generations\n",
    "master_table = master_table.append(mols_for_pd)\n",
    "master_table = master_table.reset_index(drop=True)\n",
    "master_table.to_csv(r'./generations/master_results_table_gen' + str(GLOBAL_GENERATION) + '.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cross-Over and Mutation Part 2 (Local)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 0: Minimum, Maximum: -5.135899999999999, 1.8064 \n",
      "\n",
      "Generation 1: Minimum, Maximum: 1.4163, 1.7013999999999991 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Cross-Over and Mutation\n",
    "mutation_rate = 0.05\n",
    "total_generation = 5000\n",
    "mating_pool = 50 # make sure this is smaller or equal to the population \n",
    "\n",
    "# generation 0\n",
    "gen_population = reproduce(mols_for_next_generation, population_size, mutation_rate)\n",
    "new_population_list = compute_logP_population(gen_population, mating_pool)\n",
    "new_population = [mol[0] for mol in new_population_list]\n",
    "Logp = [logP[1] for logP in new_population_list]\n",
    "minimum_LogP = min(Logp)\n",
    "maximum_LogP = max(Logp)\n",
    "\n",
    "# print(\"Generation 0: \\n{}\".format(Logp))\n",
    "print(\"Generation 0: Minimum, Maximum: {}, {} \\n\".format(minimum_LogP, maximum_LogP))\n",
    "\n",
    "# generation 1 and above\n",
    "for gen in range(1, total_generation):\n",
    "    gen_population = reproduce(new_population, population_size, mutation_rate)\n",
    "    new_population_list = compute_logP_population(gen_population, mating_pool)\n",
    "    \n",
    "    new_population = [mol[0] for mol in new_population_list]\n",
    "    Logp = [logP[1] for logP in new_population_list]\n",
    "    minimum_LogP = min(Logp)\n",
    "    maximum_LogP = max(Logp)\n",
    "    \n",
    "    print(\"Generation {}: Minimum, Maximum: {}, {} \\n\".format(gen, minimum_LogP, maximum_LogP))\n",
    "    \n",
    "    if (min(Logp) >= 1.35 and max(Logp) <= 1.8):\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "population_co_after_train = [Chem.MolToSmiles(mol) for mol in new_population] # Convert to smile\n",
    "population_co_after_train = validate_mols(population_co_after_train) # Validate molecule\n",
    "population_co_after_train = list(dict.fromkeys(population_co_after_train)) # Remove Duplicate in the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<rdkit.Chem.rdchem.Mol at 0x195744180e0>,\n",
       " <rdkit.Chem.rdchem.Mol at 0x1957441b190>,\n",
       " <rdkit.Chem.rdchem.Mol at 0x1957441b030>,\n",
       " <rdkit.Chem.rdchem.Mol at 0x1957441bad8>,\n",
       " <rdkit.Chem.rdchem.Mol at 0x1957441bfa8>]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_population[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Append the cross-over result to the mols for export\n",
    "# population_co_after_train: This is from the local cross-over\n",
    "# mols_for_next_generation: This is from the generated 5k data\n",
    "# mols_for_export: Combine mols_for_next_generation + population_co_after_train\n",
    "\n",
    "mols_for_export = mols_for_next_generation + population_co_after_train\n",
    "\n",
    "mols_crossover = population_co_after_train\n",
    "mols_transfer_learning = mols_for_next_generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Change the variables from mols_for_export -> valid_smiles\n",
    "\n",
    "# valid_smiles = list(set(mols_for_export))\n",
    "# len(valid_smiles)\n",
    "\n",
    "valid_smiles_crossover = list(set(mols_crossover))\n",
    "valid_smiles_transfer_learning = list(set(mols_transfer_learning))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "# take the valid smiles from above and run them through process \n",
    "# to add to tracking table and to generate next PyRx testing data\n",
    "\n",
    "# valid_smiles -> mols_for_next_generation\n",
    "# mols_for_next_generation = validate_mols([Chem.MolToSmiles(mol) for mol in valid_smiles])\n",
    "\n",
    "mols_crossover = validate_mols([Chem.MolToSmiles(mol) for mol in valid_smiles_crossover])\n",
    "mols_transfer_learning = validate_mols([Chem.MolToSmiles(mol) for mol in valid_smiles_transfer_learning])\n",
    "\n",
    "# Get the master table from previous generations and append it \n",
    "master_table = pd.read_csv('./generations/master_results_table_gen' + str(GLOBAL_GENERATION-1) +'.csv',sep=',')\n",
    "\n",
    "# dropping ALL duplicate values \n",
    "master_table.drop_duplicates(subset =\"smile\", keep = False, inplace = True) \n",
    "\n",
    "# Add to the tracking table\n",
    "# new_mols_to_test = append_to_tracking_table(master_table,mols_for_next_generation, 'generated', GLOBAL_GENERATION)\n",
    "# mols_for_pd = new_mols_to_test[0]\n",
    "# mols_for_export = new_mols_to_test[1]\n",
    "\n",
    "\n",
    "# Add cross-over molecule to the master table\n",
    "new_mols_to_test_crossover = append_to_tracking_table(master_table,mols_crossover, 'genetic_algorithm', GLOBAL_GENERATION)\n",
    "mols_for_pd_crossover = new_mols_to_test_crossover[0]\n",
    "mols_for_export_crossover = new_mols_to_test_crossover[1]\n",
    "\n",
    "# Add transfer learning molecule to the master table\n",
    "new_mols_to_test_transfer_learning = append_to_tracking_table(master_table,mols_transfer_learning, 'generated', GLOBAL_GENERATION)\n",
    "mols_for_pd_transfer_learning = new_mols_to_test_transfer_learning[0]\n",
    "mols_for_export_transfer_learning = new_mols_to_test_transfer_learning[1]\n",
    "\n",
    "# Join cross-over molecule with transfer learning\n",
    "mols_for_pd = mols_for_pd_crossover.append(mols_for_pd_transfer_learning)\n",
    "mols_for_export = mols_for_export_crossover + mols_for_export_transfer_learning\n",
    "\n",
    "# Remove Duplication\n",
    "# mols_for_export = [Chem.MolToSmiles(mol) for mol in mols_for_export]\n",
    "# print(\"Before: \", len(mols_for_export))\n",
    "\n",
    "# mols_for_export = set(mols_for_export)\n",
    "# print(\"After: \", len(mols_for_export))\n",
    "\n",
    "# mols_for_export = [Chem.MolFromSmiles(mol) for mol in mols_for_export]\n",
    "\n",
    "\n",
    "# Save the whole details in new table with current generations\n",
    "master_table = master_table.append(mols_for_pd)\n",
    "master_table = master_table.reset_index(drop=True)\n",
    "master_table.to_csv(r'./generations/master_results_table_gen' + str(GLOBAL_GENERATION) + '.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20965, 7)"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "master_table.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2729"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(mols_for_export)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dropping ALL duplicate values \n",
    "master_table.drop_duplicates(subset =\"smile\", keep = False, inplace = True) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ok\n"
     ]
    }
   ],
   "source": [
    "write_gen_to_sdf(mols_for_export, GLOBAL_GENERATION, 6000)\n",
    "print('ok')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}