Haunted_Mansion.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Haunted Mansion\n",
    "\n",
    "Welcome to the haunted mansion, a 3 dimensional array for an agent to learn to navigate. The agent will start out in position (0,0,0) and it will try to exit at point (size,size,size). I will start with a relatively small size and adjust it if needed.\n",
    "\n",
    "### The agent will encounter a few things throughout its journey through the haunted mansion. \n",
    "- It will encounter monsters sporadically\n",
    "    - There will be \"hot spots\", maybe 3x3x3 areas where monsters are likely to spawn, and in the center of themis lots of treasure. \n",
    "    \n",
    "- It will encounter treasure throughout its adventure\n",
    "    - The treasure will be the *value* metric of the agent. The immediate reward will be the steps that it takes. \n",
    "    \n",
    "\n",
    "### Two policies\n",
    "1. We'll try to design a policy where the agent wants to get *out* of the haunted mansion as quickly as it can. \n",
    "\n",
    "2. We'll also try a \"hero\" policy where it collects as much treasure as it can and gains strength (through slaying monsters) \n",
    "\n",
    "### Strength\n",
    "The agent will have a strength that determines whether it will die or win against a given monster. The monsters will spawn probabilistically in a given \"hot spot\" and they will have strengths as well. If the agent wins (ie. agent strength > monster strength), it gains strength. If they are equally matched, it is a coin flip. \n",
    "\n",
    "### Treasure \n",
    "Treasure will be the value function of the environment. There will be more treasure available inside of the monster zones. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's first set up the environment and the agent\n",
    "\n",
    "# importing scipy.stats for normal distribution for monster spawning probabilities\n",
    "import scipy.stats\n",
    "\n",
    "tre = 0\n",
    "mon = 1\n",
    "\n",
    "class Environment: \n",
    "    def __init__(self, size, hotspots=[(3,3,3,3)]):\n",
    "        self.size = size\n",
    "        self.hotspot_prob_cache3x3 = self.gen_hotspot_prob_matrix(3)\n",
    "        self.hotspot_prob_cache5x5 = self.gen_hotspot_prob_matrix(5)\n",
    "        self.hotspots = hotspots\n",
    "        self.map = self.generate_map(size, hotspots)\n",
    "        \n",
    "    def generate_map(self, map_size, hotspots):\n",
    "        env = np.empty(shape=(map_size, map_size, map_size, 2))\n",
    "        \n",
    "        for i in range(map_size):\n",
    "            for j in range(map_size):\n",
    "                for k in range(map_size):\n",
    "                    treasure = 0\n",
    "                    monster = 0\n",
    "                    env[i,j,k] = [treasure, monster]\n",
    "                    \n",
    "        for (x, y, z, s) in hotspots:\n",
    "            r = s // 2\n",
    "            env[x-r:x+r+1, y-r:y+r+1, z-r:z+r+1] = self.populate_hotspot((x, y, z), s)\n",
    "        \n",
    "        return env\n",
    "            \n",
    "            \n",
    "    def populate_hotspot(self, loc, hotspot_size):\n",
    "        r = hotspot_size // 2\n",
    "        x, y, z = loc\n",
    "        \n",
    "        hotspot = np.empty(shape=(hotspot_size, hotspot_size, hotspot_size, 2))\n",
    "        \n",
    "        for i in range(hotspot_size):\n",
    "            for j in range(hotspot_size):\n",
    "                for k in range(hotspot_size):\n",
    "                    # treasure is multiplied by the probability of a monster\n",
    "                    if hotspot_size == 3:\n",
    "                        treasure = self.hotspot_prob_cache3x3[i,j,k] * 50\n",
    "                    elif hotspot_size == 5:\n",
    "                        treasure = self.hotspot_prob_cache5x5[i,j,k] * 50\n",
    "                    monster = np.random.randint(8)+1       # random strength 1 to 8\n",
    "                    hotspot[i,j,k] = [treasure, monster]\n",
    "        \n",
    "        return hotspot\n",
    "    \n",
    "    \n",
    "    def regenerate_hotspots(self):\n",
    "        for (x, y, z, s) in self.hotspots:\n",
    "            r = s // 2\n",
    "            self.map[x-r:x+r+1, y-r:y+r+1, z-r:z+r+1] = self.populate_hotspot((x, y, z), s)\n",
    "            \n",
    "        \n",
    "        \n",
    "    \n",
    "    # the distance from the center of the hotspot determines\n",
    "    # how likely a monster is to spawn there - there is always\n",
    "    # a monster in the center\n",
    "    def gen_hotspot_prob_matrix(self, size):\n",
    "        \n",
    "        arr = np.zeros((size,size,size))\n",
    "        mid = (size//2, size//2, size//2)\n",
    "        xh, yh, zh = mid\n",
    "        \n",
    "        # generate probabilities at given relative locations\n",
    "        for i in range(size):\n",
    "            for j in range(size):\n",
    "                for k in range(size):\n",
    "                    dist = np.sqrt((i-xh)**2 + (j-yh)**2 + (k-zh)**2)\n",
    "                    gen_prob = scipy.stats.norm(0, size/2).cdf(-dist)\n",
    "                    arr[i,j,k] = gen_prob\n",
    "        \n",
    "        # always a monster in the center!\n",
    "        arr[mid] = 1\n",
    "                    \n",
    "        return arr\n",
    "        \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[0.12410654 0.17288929 0.12410654]\n",
      "  [0.17288929 0.25249254 0.17288929]\n",
      "  [0.12410654 0.17288929 0.12410654]]\n",
      "\n",
      " [[0.17288929 0.25249254 0.17288929]\n",
      "  [0.25249254 1.         0.25249254]\n",
      "  [0.17288929 0.25249254 0.17288929]]\n",
      "\n",
      " [[0.12410654 0.17288929 0.12410654]\n",
      "  [0.17288929 0.25249254 0.17288929]\n",
      "  [0.12410654 0.17288929 0.12410654]]]\n"
     ]
    }
   ],
   "source": [
    "env = Environment(6, hotspots=[(3,3,3,3)])\n",
    "\n",
    "# here's the probability of a monster spawning in a given 3x3x3 hotspot\n",
    "# note there will always be a monster in the middle of the hotspot\n",
    "# (this is where the most treasure is!)\n",
    "print(env.hotspot_prob_cache3x3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]\n",
      "\n",
      " [[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]\n",
      "\n",
      " [[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 8. 5. 1. 0.]\n",
      "  [0. 0. 2. 3. 7. 0.]\n",
      "  [0. 0. 5. 6. 6. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]\n",
      "\n",
      " [[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 4. 4. 5. 0.]\n",
      "  [0. 0. 8. 8. 2. 0.]\n",
      "  [0. 0. 3. 8. 1. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]\n",
      "\n",
      " [[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 7. 3. 4. 0.]\n",
      "  [0. 0. 5. 4. 3. 0.]\n",
      "  [0. 0. 2. 6. 4. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]\n",
      "\n",
      " [[0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]\n",
      "  [0. 0. 0. 0. 0. 0.]]]\n"
     ]
    }
   ],
   "source": [
    "# and here we can see the \"hot spot\" on a small map\n",
    "# the numbers are the strengths of the monsters in that area\n",
    "print(env.map[:,:,:, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Agent\n",
    "Now we have to design an agent.\n",
    "\n",
    "An agent has actions that respond to the environment. The actions here will be up, down, left, right, forward, and backward. This first Agent we're going to try to get him to simply get to the exit as fast as possible. This policy should be relatively easy to implement - we're going to just take the distance formula from the position the agent is at, and if it steps in the direction of the exit, we reward it\n",
    "\n",
    "What will be interesting is when it encounters a hotspot and gets negative reward for dying, will it tend to avoid the area? \n",
    "\n",
    "### Bellman Equation\n",
    "The Bellman equation, used to update the policies at each step:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Agent:\n",
    "    def __init__(self, environment):\n",
    "        self.actions = [0,1,2,3,4,5]\n",
    "        self.strength = 2\n",
    "        self.treasure = 0.0\n",
    "        self.environment = environment\n",
    "        self.env_size = environment.size\n",
    "        self.final_pos = (self.environment.size-1, self.environment.size-1, self.environment.size-1)\n",
    "        self.q_table = self.make_q_table(environment)\n",
    "    \n",
    "    def act(self, position):\n",
    "        x, y, z = position\n",
    "        \n",
    "        # might have to change this to pick randomly\n",
    "        action = self.choose_action(position)\n",
    "        \n",
    "        # update position\n",
    "        if action == 0: # up\n",
    "            new_pos = (x,y,z+1)\n",
    "        elif action == 1: # down\n",
    "            new_pos = (x,y,z-1)\n",
    "        elif action == 2: # left\n",
    "            new_pos = (x-1,y,z)\n",
    "        elif action == 3: # right\n",
    "            new_pos = (x+1,y,z)\n",
    "        elif action == 4: # forward\n",
    "            new_pos = (x,y+1,z)\n",
    "        elif action == 5: # backward\n",
    "            new_pos = (x,y-1,z)\n",
    "        \n",
    "        new_x, new_y, new_z = new_pos\n",
    "        \n",
    "        # check if agent went off the map\n",
    "        if new_x < 0 or new_y < 0 or new_z < 0 or new_x >= self.env_size or new_y >= self.env_size or new_z >= self.env_size:\n",
    "            reward = -5\n",
    "            new_pos = position\n",
    "            return action, new_pos, reward, False\n",
    "        \n",
    "        reward = 1\n",
    "        \n",
    "        # check for monsters and simulate interaction \n",
    "        monster = self.environment.map[new_pos][1]\n",
    "        if monster:\n",
    "            if self.strength > monster or (self.strength == monster and np.random.uniform() <= 0.5):\n",
    "                self.treasure += self.environment.map[new_pos][0]\n",
    "                self.strength += 1\n",
    "            else:\n",
    "                reward = -5\n",
    "                return action, new_pos, reward, True\n",
    "        \n",
    "        return action, new_pos, reward, False\n",
    "            \n",
    "        \n",
    "        # Function for choosing the action for the agent\n",
    "    def choose_action(self, position):\n",
    "\n",
    "        # Selection of the action - 90 % according to the epsilon == 0.9\n",
    "        # Choosing the best action\n",
    "        if np.random.uniform() < 0.9 and not (np.argmin(self.q_table[position]) == np.argmax(self.q_table[position])):\n",
    "            state_action = self.q_table[position]\n",
    "            action = np.argmax(state_action)\n",
    "        else:\n",
    "            # Choosing random action - left 10 % for choosing randomly\n",
    "            action = np.random.choice(self.actions)\n",
    "        \n",
    "        return action\n",
    "\n",
    "    \n",
    "    \n",
    "    def learn(self, episodes=1000, learning_rate=0.9):\n",
    "        \n",
    "        self.learning_rate = learning_rate\n",
    "        \n",
    "        # reward decay\n",
    "        self.gamma = 0.1\n",
    "        \n",
    "        # reset q table\n",
    "        self.q_table = self.make_q_table(self.environment)\n",
    "        \n",
    "        for e in range(episodes):\n",
    "            position = (0,0,0)\n",
    "            self.strength = 0\n",
    "            if e % 100 == 0:\n",
    "                print('episode {}'.format(e))\n",
    "            while position != self.final_pos:\n",
    "                action, new_pos, reward, dead = self.act(position)\n",
    "                if new_pos == self.final_pos:\n",
    "                    self.q_table[position][action] += 50\n",
    "                else:\n",
    "                    self.q_table[position][action] += learning_rate * ((reward + self.gamma * max(self.q_table[new_pos]) - self.q_table[position][action]))\n",
    "                # print(learning_rate, reward, self.gamma, max(self.q_table[new_pos]))\n",
    "                if dead:\n",
    "                    position = (0,0,0)\n",
    "                    break\n",
    "                \n",
    "                position = new_pos\n",
    "                \n",
    "    def run(self):\n",
    "        xy_run = np.zeros((self.env_size, self.env_size))\n",
    "        xz_run = np.zeros((self.env_size, self.env_size))\n",
    "        yz_run = np.zeros((self.env_size, self.env_size))\n",
    "        \n",
    "        xy_run[0,0] = 1\n",
    "        xz_run[0,0] = 1\n",
    "        yz_run[0,0] = 1\n",
    "        \n",
    "        position = (0,0,0)\n",
    "\n",
    "        while position != self.final_pos:\n",
    "            action, new_pos, reward, dead = self.act(position)\n",
    "            # self.q_table[position][action] += reward\n",
    "            \n",
    "            x, y, z = new_pos\n",
    "            xy_run[x,y] = 1\n",
    "            xz_run[x,z] = 1\n",
    "            yz_run[y,z] = 1\n",
    "\n",
    "            if dead:\n",
    "                position = (0,0,0)\n",
    "                break\n",
    "                \n",
    "            position = new_pos\n",
    "        \n",
    "        return xy_run, xz_run, yz_run\n",
    "            \n",
    "    \n",
    "    # the q table is a dictionary containing the reward at each \n",
    "    # position in the environment\n",
    "    def make_q_table(self, environment):\n",
    "        \n",
    "        q_table = dict()\n",
    "        width, height, depth, size = environment.map.shape\n",
    "        \n",
    "        for i in range(width):\n",
    "            for j in range(height):\n",
    "                for k in range(depth):\n",
    "                    # index:action\n",
    "                    # 0:up, 1:down, 2:left, 3:right, 4:fwd, 5:bkwd\n",
    "                    q_table[(i, j, k)] = [0,0,0,0,0,0]\n",
    "        \n",
    "        return q_table\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "episode 0\n",
      "episode 100\n",
      "episode 200\n",
      "episode 300\n",
      "episode 400\n",
      "episode 500\n",
      "episode 600\n",
      "episode 700\n",
      "episode 800\n",
      "episode 900\n"
     ]
    }
   ],
   "source": [
    "learner6x6 = Agent(env)\n",
    "learner6x6.learn()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{(0, 0, 0): [1.1111111114911687, -4.888888888852876, -4.88888888885404, 1.111111110992801, 1.1111111114655698, -4.888888888855207], (0, 0, 1): [1.111111111145298, 1.1111111111485183, -4.888888888531959, 1.1111111111450027, 1.1111111149168595, -4.8888888885513495], (0, 0, 2): [0, 1.1111111114540295, -4.400000000981848, 0, 1.0000000002996476, 0], (0, 0, 3): [0.9, 1.09989, 0, 1.1111111111111112, 1.1078857751751001, -4.401436851000001], (0, 0, 4): [1.1111099504801822, 0, 0, 0, 0, -4.4190000000000005], (0, 0, 5): [-4.5, 1.1111077107547997, 0, 0.9, 0.9, 0], (0, 1, 0): [1.1111111148086166, -4.884999999745851, 0, 0, 1.00000000059506, 0], (0, 1, 1): [1.1111111114506524, 1.1111111114752377, -4.888888885109923, 1.1111111114571046, 1.1111111492040877, 1.111111111479972], (0, 1, 2): [0, 1.111111114660853, 0, 0, 0, 0], (0, 1, 3): [0, 0, 0, 0, 0, 1.110998563149], (0, 1, 4): [0, 0, 0, 0, 0, 0.9970460999999999], (0, 1, 5): [0, 0.9, 0, 0, 0, 0], (0, 2, 0): [1.1111111483339784, -4.88884, -4.840001, 1.11, 1.109999999999999, 1.1110995], (0, 2, 1): [1.1111111142069976, 1.111111114621916, -4.888888857411537, 1.1111111145566133, 1.1111114924863947, 1.1111111147642558], (0, 2, 2): [1.111032041705741, 1.1111000134453788, -4.883999999861551, 1.109999999999997, 1.1111111438793086, 1.1111109999661957], (0, 2, 3): [1.1111110895752934, 0, -4.5, 0, 0, 0], (0, 2, 4): [1.08, 0, -4.4, 1.1111111111111112, 0, 0], (0, 2, 5): [0, 1.1, 0, 0, 0, 0], (0, 3, 0): [1.0000003046480288, -4.399999975475419, -4.4, 1.1111114361521819, 1.0000002295274777, 1.0000000009924803], (0, 3, 1): [1.1111111460294256, 1.1111111383013308, -4.888888535906483, 1.111114930327646, 1.1111111445081938, 1.111111148773219], (0, 3, 2): [1.106999476679629, 1.111111483617338, -4.888839972427541, 1.1111000001516493, 1.1111000001732227, 1.1000000000657952], (0, 3, 3): [0, 0, 0, 0, 0, 1.1099995929730446], (0, 3, 4): [0, 0, 0, 0, 0, 0], (0, 3, 5): [-4.40019909, 0, 0, 0.9994599000000001, 1.11088989, 0], (0, 4, 0): [1.1111111392967616, -4.888888584592104, -4.888883770472523, 1.111114534273441, 1.111098889007748, 1.1111107981778314], (0, 4, 1): [1.1111111100271684, 1.1111114502564252, -4.88884, 1.110997775077751, 1.1098989999999997, 1.1], (0, 4, 2): [0, 0, -4.399999999826744, 0, 0, 1.1111111140057066], (0, 4, 3): [0, 0, 0, 1.1109999995963107, 0, 0], (0, 4, 4): [0, 0, -4.40000001, 1.1111111, 0, 0], (0, 4, 5): [0, 0, -4.40001, 0, 1.1111111, 0], (0, 5, 0): [1.11111111111111, 0, -4.5, 0, -4.840000001022501, 0], (0, 5, 1): [0, 0, -4.4, 1.0, 0, 1.1111111111111112], (0, 5, 2): [0, 0, 0, 0, 0, 0], (0, 5, 3): [1.11111111, 0, -4.4000000009999995, 0, 0, 0], (0, 5, 4): [1.1111111111111112, 1.1111111009999999, -4.88884, 1.1111111, -4.88884, 1.1097999], (0, 5, 5): [-4.888850000000001, 1.1111111111111112, -4.888895000000001, 1.11111111111, -4.888888888884001, 1.1109879], (1, 0, 0): [1.1111111111182372, 0, 1.0000000000213387, 0.989099999918927, 0, -4.5], (1, 0, 1): [0.9999999890908041, 0, 0, 0, 1.1111111114702727, -4.399999999994226], (1, 0, 2): [0, 1.1111110989968542, 0, 0.9999780543473341, 0, 0], (1, 0, 3): [0, 0.999999890901716, 1.1111111111111112, 0, 0, 0], (1, 0, 4): [0, 1.1089575293410001, 0, 0, 0, 0], (1, 0, 5): [0, 0.9810000000000001, 0, 0, 0, 0], (1, 1, 0): [0, -4.4001090000025025, 1.1110990001486212, 0, 0, 0], (1, 1, 1): [1.0899998800545734, 1.110671100000953, 1.1111111147508412, 1.08, 1.0, 0], (1, 1, 2): [1.1111111111111112, 1.1000000000002015, 0, 0, 0, 1.0989088279894461], (1, 1, 3): [1.0988999960034143, 1.1111111111111112, 0.998987068341, 1.0799999999991068, 0.9, 0], (1, 1, 4): [0, 1.1109999995962143, 0, 0.9810000000000001, 0, 0], (1, 1, 5): [0, 0, 0, 0, 0, 0], (1, 2, 0): [1.1111111146176469, -4.5, 0.9999999999999999, 1.0, 1.0, 0.9998909999974973], (1, 2, 1): [1.11111111111098, 1.111111111111099, 1.111111147338777, 1.1111111111110987, 1.1111111111111112, 1.1111111114290815], (1, 2, 2): [0, 1.1111111139922794, 0, 0, 0, 0], (1, 2, 3): [0, 0, 0, 0, 0, 1.0], (1, 2, 4): [1.10349, 0, 1.1111111111111112, 0, 1.0710000000000002, 1.1087999992731856], (1, 2, 5): [0, 0, 0, 0, 1.10349, 0], (1, 3, 0): [1.111001536658595, -4.399999999979021, 0, 0.9999999999882883, 1.1111144692638266, 1.1], (1, 3, 1): [1.1111111146504966, 1.1111114170602845, 1.1111114076256523, 1.111149318568792, 1.111139264674432, 1.1111111137347325], (1, 3, 2): [0.9900000000000001, 0, 1.1111111473611677, 0, 0, 0], (1, 3, 3): [0, 0, 0, 0, 1.11, 0], (1, 3, 4): [0, 0, 0, 0, 1.1109, 0], (1, 3, 5): [0, 0, 1.11065301, 0, 0, 0], (1, 4, 0): [1.1111109809777486, -4.888400000000001, 1.1111113342011818, 1.1111461670999696, 1.1111100001267848, 1.1109999980988383], (1, 4, 1): [0.9990000000000001, 0, 0, 1.0999999999624845, 1.1114066666170417, 0], (1, 4, 2): [0, 1.1100000015724734, 0, 0, 0, 0], (1, 4, 3): [1.1111111111111112, 0.9900000000000001, 1.1105999989100412, -4.5, 0.9999999051824151, 1.0890000000000002], (1, 4, 4): [1.1109978162053005, 1.1111111111111112, 1.11108969, -4.9995, 1.0, 1.09962], (1, 4, 5): [-4.4010909, 0, 0, 1.1111111111109895, 0, 0.9997668900000001], (1, 5, 0): [1.1113505153953958, -4.884, 1.1111111108862461, 1.1111109999999838, -4.888840000000001, 1.1111000016482873], (1, 5, 1): [1.1141327206671756, 1.111111, 1.11089999999999, 1.1111111111111112, -4.888884, 1.1111109999997775], (1, 5, 2): [0, 0, 0, 1.144768036300669, -4.839999937724178, 0.9], (1, 5, 3): [0, 1.1135927658476887, 0, 0, 0, 1.0], (1, 5, 4): [1.1111111111111112, 1.1112847251717795, 1.1111111111, 1.111099980946851, -4.88888888888885, 1.1111], (1, 5, 5): [-4.888885, 1.1111111111111112, 1.111111089998924, 1.1110980996022617, -4.888888888888889, 1.111111107874784], (2, 0, 0): [1.09889999989992, -4.5, 0, 0, 0, 0], (2, 0, 1): [0, 0, 1.1110000000010898, 0, 0, 0], (2, 0, 2): [0, 0, 1.1110867161433977, 0, 0, 0], (2, 0, 3): [0, 0, 0, 0, 0, 0], (2, 0, 4): [0, 0, 1.0880617764069, 0, 0, 0], (2, 0, 5): [0, 0, 0, 0, 0, 0], (2, 1, 0): [0, 0, 0, 0, 0, 0.9], (2, 1, 1): [0, 0, 1.10000000004827, 0, 0, 0], (2, 1, 2): [0, 0, 0, 0, 0, 0.9997903623188045], (2, 1, 3): [0, 0, 1.1099999999999008, 0, 0, 0], (2, 1, 4): [0, 0, 0, 0, -4.5, 1.0710000000000002], (2, 1, 5): [0, 0, 0, 0, 0, 0], (2, 2, 0): [0, -4.400000000000199, 1.111111111444656, 0.9996389828987173, 1.0998711711728983, 0.9], (2, 2, 1): [-4.5, 1.1111111111424388, 1.1, 0, 0.9999999810000001, 0], (2, 2, 2): [0, 0, 0, 0, 0, 0], (2, 2, 3): [0, 0, 0, 0, 0, 0], (2, 2, 4): [0, 0, 0, 0, 0, 0], (2, 2, 5): [0, 0, 0, 0, 0, 0], (2, 3, 0): [1.1111466286254224, -4.400000011711727, 0, 1.0000000249893102, 0, 1.0], (2, 3, 1): [-5.0, 1.1111145707457672, 1.1111148271253934, 1.1114939957895733, 1.111114789600326, 1.1111111111140048], (2, 3, 2): [0, 0, 0, 0, 0, 0], (2, 3, 3): [0, 0, 0, 0, 0, 0], (2, 3, 4): [0, 0, 0, 0, 0, 0], (2, 3, 5): [0, -4.5, 0, 0, 0, 0], (2, 4, 0): [1.1111135114024804, -4.888888769222197, 1.1111139034491775, 1.1114634019184018, 1.1, 1.1111127972963413], (2, 4, 1): [-4.99995, 1.1000287054204239, 1.1100000008607884, 1.1031627325655213, 1.1111, 1.1111485973940918], (2, 4, 2): [0, 0, 0, 0, 0, 0], (2, 4, 3): [0, 0, 0, 0, 0, 0], (2, 4, 4): [0, 0, 0, 0, 0, 0], (2, 4, 5): [0, 0, 0.999999999989054, 1.1111111111111112, 1.089999999996385, 0], (2, 5, 0): [1.1, -4.88884, 1.1111294080003158, 1.1108798244642824, -4.88884, 1.1], (2, 5, 1): [1.110999962053228, 1.1111111111111112, 1.1111, 1.111089998924031, -4.888888884, 1.1098828981261901], (2, 5, 2): [0, 0, 0, 1.4633719698421832, -4.3999979485008875, 0], (2, 5, 3): [0, 0, 0, 0, 0, -4.5], (2, 5, 4): [0.9999999999999964, 0, 1.1111111130421754, 0, 0, 0], (2, 5, 5): [-4.5, 1.1111111112345806, 0, 0, -4.5, 1.0], (3, 0, 0): [1.1111111111111112, -4.888400000403687, 0, 0.9, 1.1087999999983154, -4.888500000533223], (3, 0, 1): [1.1, 1.1111111111111112, 1.0889999999009108, 0.998563149, 0, 0], (3, 0, 2): [0, 1.1110899411575588, 1.1051018116345588, 1.1111066069727176, 1.1111111111111112, -4.884], (3, 0, 3): [0, 0, 0, 0, 0, 0], (3, 0, 4): [1.10349, 0, 0, 0, 0, 0], (3, 0, 5): [0, 0, 0, 0, 1.1043, 0], (3, 1, 0): [0, 0, 0, 0, 0, 1.1109999999990643], (3, 1, 1): [1.1111111111111112, 0, 0, 0, 0, 0], (3, 1, 2): [0.9988199999999999, 1.1109942999948181, 0.9, 1.1099889622259917, 0, 1.1111111111111112], (3, 1, 3): [1.1110978899999, 0, 0.9989999999999107, 0, -4.5, 0], (3, 1, 4): [0.9999710991, 0, 0.99, 1.11111111111, -4.5, 0.99639], (3, 1, 5): [-4.5, 1.1111078997900001, 0, 0, 0, 0], (3, 2, 0): [0, 0, 0, 1.0, 1.1111112464814374, 0.9899999999157729], (3, 2, 1): [-4.9999999999995, 1.1111111111110912, 1.1111111111109, 1.11111109, 1.1114775425110694, 1.1111111111111107], (3, 2, 2): [0, 0, 0, 0, 0, 0], (3, 2, 3): [0, 0, 0, 0, 0, 0], (3, 2, 4): [0, 0, 0, 0, 0, 0], (3, 2, 5): [0, 0, 0, 0, 0, 1.0999682098199999], (3, 3, 0): [1.0, 0, 1.1099999999882648, 1.111398073073812, 0, 0.9999999999999997], (3, 3, 1): [-5.0, 1.1111375141906241, 1.1111485418244393, 1.1145882716480389, 1.1149423793229862, 1.1111466294601131], (3, 3, 2): [0, 0, 0, 0, 0, 0], (3, 3, 3): [0, 0, 0, 0, 0, 0], (3, 3, 4): [0, 0, 0, 0, 0, 0], (3, 3, 5): [-4.840000000000001, -4.95, 0.9, 1.1111111111111112, 1.1, 1.07997399], (3, 4, 0): [1.11484757584154, 0, 0, 1.1035100823645525, 0, 1.00000015294901], (3, 4, 1): [-4.999999999999995, 1.111450080972265, 1.1111147352507424, 1.1494749696853739, 1.1113430339574641, 1.111490746318294], (3, 4, 2): [0, 0, 0, 0, 0, 0], (3, 4, 3): [0, 0, 0, 0, 0, 0], (3, 4, 4): [0, 0, 0, 0, 0, 0], (3, 4, 5): [-4.888884, -4.999999995, 1.1111111111060001, 1.1111111111111112, 1.11111111111109, 1.1111109998180915], (3, 5, 0): [0, -4.4000001003758, 1.1111109995824449, 0, 0, 0], (3, 5, 1): [1.1111342666278319, 1.1098976166844183, 1.111, 1.1111209034733591, -4.888817482544017, 1.1144109611911064], (3, 5, 2): [0.9998999999991577, 1.1113450160543676, 0, 4.716782651668286, 0, -4.5], (3, 5, 3): [1.1130701119293773, 0, 0.9, 0, 0, -4.5], (3, 5, 4): [1.1111111111111112, 1.111111083, 1.111111111110998, 1.1422437801592316, -4.888888884, -4.99999999999995], (3, 5, 5): [-4.88888888888885, 1.11400168359579, 1.1111111109865837, 1.11111111111, -4.88888888884, 1.1111111], (4, 0, 0): [1.1110991559001087, 0, 0, 0, 0.9985040999999999, -4.5], (4, 0, 1): [0.9999999838047717, 1.1111066257053908, 0, 0, 0, 0], (4, 0, 2): [0.9999999989870684, 0, 1.0, 1.1111111111093117, 0.9999998996219763, 0], (4, 0, 3): [0, 0, 0, 1.0, 1.1111111111111112, 0], (4, 0, 4): [0, 1.111111106974032, 1.0710000000000002, 0, 0, 0], (4, 0, 5): [0, 1.0999590438610765, 0, 0, 0, 0], (4, 1, 0): [1.1094362242765161, 0, 0, 0, 0, 0], (4, 1, 1): [0, 0, 0, 1.111095934111052, 0, 0], (4, 1, 2): [1.0, 0, 0, 0, 0, 1.1111110999401825], (4, 1, 3): [1.1111111111111112, 1.1110679435379058, 1.1110730120999999, 1.11111, -4.999999995, 1.1110999878346806], (4, 1, 4): [1.111109989997999, 1.1111111111111112, 1.1111097899790001, 1.111, -4.9999995, 1.1111100453383402], (4, 1, 5): [-4.400000000000101, 1.0999999999999943, 0, 1.1111111111111112, 0.9, 0], (4, 2, 0): [1.1111111111111112, -4.888888885000001, 0.9999963898118858, 1.1051999999959372, 1.0, 1.07829], (4, 2, 1): [-4.995, 1.1111111111111112, 1.1111004131049602, 0, 1.1111, 1.10881230831], (4, 2, 2): [0, 0, 0, 0, 0, 0], (4, 2, 3): [0, 0, 0, 0, 0, 0], (4, 2, 4): [0, 0, 0, 0, 0, 0], (4, 2, 5): [0, 0, 0, 1.1, 0, 0], (4, 3, 0): [1.1111301288451687, -4.888850000000001, 1.1111111109979082, 1.1144097919180835, 1.088999999999987, 1.11111111], (4, 3, 1): [-4.9999999999949996, 1.1111121573081344, 1.1113580453426317, 1.1485057446542348, 1.11, 1.111], (4, 3, 2): [0, 0, 0, 0, 0, 0], (4, 3, 3): [0, 0, 0, 0, 0, 0], (4, 3, 4): [0, 0, 0, 0, 0, 0], (4, 3, 5): [-4.839947873684509, -4.5, 1.1111111111111112, 1.1373081466250963, 1.1111, 0.9900000000000001], (4, 4, 0): [1.1475340277006454, -4.5, 0, 0, 0, 0], (4, 4, 1): [-4.99999999995, 1.1144165011624358, 1.11462432394281, 1.4956832988229842, 1.114479467998714, 1.1147507513439627], (4, 4, 2): [0, 0, 0, 0, 0, 0], (4, 4, 3): [0, 0, 0, 0, 0, 0], (4, 4, 4): [0, 0, 0, 0, 0, 0], (4, 4, 5): [-4.888888400000001, -4.9995, 1.1111111111111112, 1.1111111109999998, 1.1111111111110001, 1.1128401854807923], (4, 5, 0): [1.111111, 0, 0, 0, -4.5, 0.9899999999998716], (4, 5, 1): [1.1110986567173515, 1.11111084, 1.1111111111111112, 1.110879, -4.888400000000001, 1.1470333914711592], (4, 5, 2): [1.4185345069403306, 1.1017601806198771, 1.111106070010658, 37.94796043487373, -4.840000000000001, -4.995], (4, 5, 3): [1.139293140874918, 4.572145200309345, 1.1089999998, 1.0, -4.884, -4.5], (4, 5, 4): [1.1111111111111112, 1.4525385826661323, 1.1, 0, 0, -4.99995], (4, 5, 5): [0, 0.9999953100000001, 1.1113397085274084, 0, 0, 0], (5, 0, 0): [0, 0, 0, 0, 0, 0], (5, 0, 1): [0, 0, 0, 0, 0, 0], (5, 0, 2): [1.1111111111111112, 0, 0, -4.400000000000002, 0.9999999997775051, -4.5], (5, 0, 3): [1.1111111111111112, 1.0999999999999999, 1.0999989870582116, -4.4, 1.0989999989748804, -4.840002505597377], (5, 0, 4): [1.1111111111111112, 1.111, 1.0999255335486375, -4.885000000000001, 1.11, -4.400000000000003], (5, 0, 5): [-4.884, 1.0, 1.0799664905071242, -4.40000000019053, 1.1111111111111112, -4.840000000000001], (5, 1, 0): [0, 0, 0, 0, 0, 0], (5, 1, 1): [1.1111100180508915, 0, 0, 0, 0, 0], (5, 1, 2): [0, 0, 0, 0, 0, 1.111111111110864], (5, 1, 3): [0, 0, 1.0, 0, 0, 1.1111111111111112], (5, 1, 4): [1.1111111111111112, 1.1111111111111112, 1.1111111111111112, -4.888888888888889, 1.1111111111111112, 1.1111111111111112], (5, 1, 5): [-4.888888888888889, 1.1111111111111112, 1.1111111111111112, -4.888888888888889, 1.1111111111111112, 1.1111111111111112], (5, 2, 0): [1.1108999999979798, 0, 0, -4.5, 0, 0], (5, 2, 1): [1.1105999935901494, 0.9998099999981819, 1.0, -4.840000000033968, 1.1111111111111112, 0.9999238516605324], (5, 2, 2): [0, 0, 0, 0, 0, 1.1110999997545883], (5, 2, 3): [1.1109999999999, 0.9999899997793297, 0, -4.4010000000008995, 0, 0], (5, 2, 4): [0, 1.11059999999973, -4.95, 0, 1.1484792112656996, 1.1111111111111112], (5, 2, 5): [-4.400000000000001, 1.1111111111111112, 0, 0, 1.09, 1.0], (5, 3, 0): [0, -4.88850465503034, 0, 0, 1.145555203425807, 0], (5, 3, 1): [1.4912393196818932, 1.109997732115173, 1.0, -4.888400000000002, 1.3819371465927501, 1.1111111111111112], (5, 3, 2): [4.935359610539851, 1.122344703358044, -4.5, -4.740462433917667, 4.215897936927436, 0], (5, 3, 3): [1.0, 0, 0, -2.848051884940241, 39.53163162583436, 0.999989999999991], (5, 3, 4): [1.1111111111111112, 4.666057271847199, 0, -4.4, 0, 1.0], (5, 3, 5): [-4.4, 1.4305284016406616, 1.0899999999736545, 0, 0, 0], (5, 4, 0): [1.4657495899022184, 0, 0, 0, 0.9, 0], (5, 4, 1): [4.964218096967612, 1.145754699845185, 1.147069289368063, -4.510502979249546, 1.1338618105634215, 1.1485358094758993], (5, 4, 2): [39.69328283283324, 1.480034424890642, -5.0, -1.6044623094471357, 36.87969543950303, 1.486728450122076], (5, 4, 3): [1.1111111111111112, 4.457044418036632, -4.999999999999999, 31.136953238262596, 387.48888888888894, 4.8229617001066964], (5, 4, 4): [1.1111111111111112, 0, -4.995, -4.4, 0, 1.3132601267368313], (5, 4, 5): [-4.884, 0, 1.1111158023795054, -4.400000000000019, 0, 1.0], (5, 5, 0): [1.0000168307171358, -4.5, 0, 0, 0, 0], (5, 5, 1): [0, 0, 1.1112981190792874, 0, -4.864399830735937, 1.3652321953292614], (5, 5, 2): [386.2466024750668, 1.006057503819901, 0.9999777439734832, 20.72644481534471, 0, 2.996400794250001], (5, 5, 3): [3870.4444444444443, 37.64731252881174, 1.441067160849358, 372.5484964062251, 370.64385239511165, 31.959888343046682], (5, 5, 4): [38750, 364.24590663801115, 1.1444262035060817, 3805.427100393744, 3825.3790836112016, 1.1111111111111112], (5, 5, 5): [0, 0, 0, 0, 0, 0]}\n"
     ]
    }
   ],
   "source": [
    "print(learner6x6.q_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a pseudo visual for what a single run thru the haunted mansion looks like - think of it like a 2d x-ray on each pair of axes.\n",
    "\n",
    "It looks like the agent is definitely avoiding the monster hot spot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 1. 1. 1. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 1. 1. 1.]] \n",
      "\n",
      " [[1. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 1.]] \n",
      "\n",
      " [[1. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 0. 0.]\n",
      " [0. 1. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 1. 1. 1.]]\n"
     ]
    }
   ],
   "source": [
    "xy, xz, yz = learner6x6.run()\n",
    "print(xy, '\\n\\n', xz, '\\n\\n', yz)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 355,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "episode 0\n",
      "episode 100\n",
      "episode 200\n",
      "episode 300\n",
      "episode 400\n",
      "episode 500\n",
      "episode 600\n",
      "episode 700\n",
      "episode 800\n",
      "episode 900\n"
     ]
    }
   ],
   "source": [
    "env10x10 = Environment(size=10, hotspots=[(2,2,2,3), (4,3,5,3)])\n",
    "learner10x10 = Agent(env10x10)\n",
    "\n",
    "learner10x10.learn()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 356,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 1. 1. 1. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]] \n",
      "\n",
      " [[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]] \n",
      "\n",
      " [[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]]\n"
     ]
    }
   ],
   "source": [
    "xy, xz, yz = learner10x10.run()\n",
    "\n",
    "xy, xz, yz = learner10x10.run()\n",
    "print(learner10x10.treasure)\n",
    "print(xy, '\\n\\n', xz, '\\n\\n', yz)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Wow\n",
    "I'm amazed at how quickly this thing can learn. It seems that once it finds a correct path, the training speeds up significantly because each run is straight to the exit."
   ]
  },
  {
   "attachments": {
    "1_raYvY38yWm4E4J3iYqv6dA.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhoAAAC7CAIAAACsBKATAAB6/UlEQVR42uxdCXxU1dW/9743eyZ7ICshhCVhTQCDoAgiAiqIirihlop73T4trRUqbbGtXd1qtWiLuBUVEapIERCoshMIJAESAgRCgJA9mcls7937/eadmZvHmyRsIZD4TkucJJM3dzn3LP+zXJExhnTSSSeddNLpwojoS6CTTjrppJOuTnTSSSeddNLViU466aSTTro60UknnXTSSSddneikk0466aSrE5100kknnXR1opNOOumkk65OdNJJJ5100klXJzrppJNOOl0cEs/p3YwxjDGSZEQpwhjpFfVtE4Z/jIkixjiweudO8Ic+JsuMYoQZ0pddJ50u9tn1HzQBEwMW/CeOISYxpvz00o8NRC9BRCDnLVUuysD0Jis66aSTTjp1tHeCZIoEgr7egPaWosgwVO1AItZt5ZaJMmQ1+peLMfTALchsPm87wkepgZDXGnctcRSYDJE+yQGOj0466XRxiIULYXVS3Y/s2bPCBlLEfPXeoysOCWaB+Sj1KdjMpRoZYyar6HH4IvpExV+ZiKgikzujd8J8PmwwoKsfRBv/iVB/hPbqfHcmSkXoKDpZibrHUEoJOZ+Nd8lei2DEpX9GRz5F1gTkPI6wHvTSSaeLKLNRWAJyHEfps+SURyhizgP1S65Zbk+1Oo+7PCclYrhE6kRA1EFjRoTXFTsyZ/a56s9XM4kSg9ApvZOAcd0/GW0cjR4bj7pH+f0VrFvKLbEjBJm+3oZK+iORNK/eeeh8+ENjLApLRpHZNyV0b0RU1JddJ50ukvmPyCrPCYZJnDEqcASNJHZIpD3F2u+e3hcSB73wsRGC3dXuY2EnbPFWjDC9nOSAeD5/dLASoe/QlN+wG8YiHXZpcdeBBxsd6L21yLEdsfZYJKkRNZ14Nvnuv3Sfrq+wTu3JrPoZDqEnTi7+e/l/KyVH4HuZ1RQ22hNtgx4bTAzipR1b2belhe+UuId7LresnAtYF5/kV9GSzERBZz7tCaUU+a0ID/KxgLPSTuRmlDImMyroeJdO5+TjBqwcxv1nxighhGDi5yhZ8r/0292IYMzQZZLEdAkIDpeHyaEWIqNM8soGUbhU3gmVKREIlajaX+EBi0voNl2wOsEBQheQ/Hr2kZs2VuqySpXjww38n+DAt+32YEwwpsj/VReROp0bTqJoi+DRo6IgOhyOioqKbt262e12SikOAqtYcVh+qDzmP1ynixRGCeOwM/9VB4sd2DjN0DTfX1pJKF7UmbehLc5VkbbxwEuik0PHw4cB1gLukDFcOnWJL0ctfvHXsLNPHAYvy7IoiiUlJWvXrkUI7du3b+jQoffeey+lfrOXJ4y0i7XXJRw7TJhWhVwagYNo0NUM7E5X8U7OpELbnti5eid87QLyOvgpl1aq8iH5LTuFAgeyQ8Z0lnzDNwIG6XfnZbnZ1yHkQrR7Z5cg5zp+9cQ76fRBl1RUVCxbtmzs2LHDhw9ftmzZc889JwjCjBkzJElS4r3kh8kPrc0ucKBZs7S5JCZsUJEEvvdvFkXCZdPchLS7oacW94wxSil8y19oXnM9QUOIvxNktFoDCYJwTtroYhMfT0DDYUw7hL1aXF4N8fPAuV8MkiAE4l58hVt7juaD1CL1ovoNbcyoXWREazzZ2q86TJe0Nv3znrvGmN2+fbvD4Rg2bBhj7JZbbhk8ePB7773n9XoFQTinZekwfugAlmjxIwP/JcrnXh7hJIwRo0w0GkRRNJoMl48KF9vd0OPWuuY1l1xcbEHoD2SZ+rcaUpdr8AeWlpbGx8ebTCawpLjd3fFgF0AHp06d6t69uyAIMBJGKelA41qLp7a+kjDy7du3FxUVYYxHjx7ds2dPWZbBGuU70ppVG4zWdoSBpsGpL9CraINdA4ZVmyVBMPdQs+mSTP/8PpQPGP68vLz8zTffvOGGG3JychhjSUlJmzdvbmpqioyMPM3ebH1ZwOYD9dMxBvvFZonQUw5RT4opkxT4gV3K6kX/IUWBI2w0GQ4Ul+R/VJgYmTBm3BiLxXI5OIhi+xoOoVwlSZLBYEAIeb1ep9Mpy7LRaAwPDyeESJIEUkwQhMbGxm+//baurg4kMiHE5/OFh4dfd911ERER/OGUUlEUlyxZIsvynXfeeam8TjXvSpIkiuK+ffs+++yzRx55xGg0+jceIRkh4eI7TzB3r9frcDjA26CUer1e9SDBEcEYwxskSfrzn/986tSpngqlpaXxN/PniKIoyzI8B/7QbreLoggSRC1iLt7iX+xoGdcKYAF4PB6n0wlLxBjzeDzwW4PBwOfeoh662PZKe82dQ1jwOicn56677oqNjWUKZlJcXJycnBwWFsYtM/igxsZGSZLMZjMskSzLoJAsFovZbAZFqx7MRT2MZxORbd9TjpRNX/v9uu8+XDfYmk2VvMpLInaCE8fBrApyouLEN0tXN51qyhyY2bNnT+DPLhKKPy0QrUK6DAZDTU3N1q1bDx065HA4vF6vJEndunWbOnVqcnKyz+cjhDQ2Nr7zzjv5+fkVFRXApjU1NT179kxJSbnmmmvUoD/oktzc3J///OeaiMWlyrIACT527Ng9e/a89dZbTz31lJJryFC7JnS1NgZZlg0GQ3Fx8T//+c+DBw86nU6r1Zqamsrf43K56urqjh49mpiY+Oqrr6alpVFK3W733LlzJ0yYwA1MwNMPHz68YMGC4uLipqamsLCw5ORk0PperzcyMnLKlClXX321etk7csE1UeJ2fCzYKMXFxe+++y5wqdVq7dmzp3rut9xyy9VXXw0sDRPnovliYzuauZ/3h/ITCvoyOzv7jTfeAONg69at+/fv/9vf/iaKoiRJ3Mn2er1LlixZvXr1qVOnJElKSUmJjo6WZdnn81FKs7Ozp02bFh8fz/GuDhZe3GXkwdqLsAf+la84cXJvYWGWMNQ/zYvoDJ39VpKmJtc1o6++//n7n3noaZfLdZlErcR25Hu1LgGbRRTFpUuXLl68ODs7e9y4cQkJCYSQ2traDz74YNq0aW+88UZOTg5CqKioKDU19fHHH//kk0/69etnNpuPHTs2efJkNdgFUu/YsWPLli2bPXt2TEyMz+dTRyzOyExtL/f5ZfjwjyaE3H///U8//fSqVatuvPFGn9stCAK6yDgyBNIppT169HjkkUfefPPNv/3tb0899dSjjz4qSRLM1+Fw1NfX/+Mf/1i3bp3X6+UCBaLxoEX4dJKTkx988MG333779ddfnzNnzt13300plSSpsbFx3bp106ZNe+aZZ+bMmaOJxHQMARzaBkp+ftKEmyOpqakPP/zw3xWaPXv2j370I1mhhoaG1atX33nnnU8//fRzzz3Ht7sDFkEdKdScsvOw3DULxR3N2traP/7xjw8++ODkyZOBbfijRFGcMGFCSkrKjBkzTCbT3LlzU1JSPAodO3bsr3/96wcffLBw4cLMzEy1j9IxvAHipW2WaC+wSxRFk9nE5NPSf8642mfjR54v06oUKiYaCLrTqxMuW9UKQJbl3//+94sXL54zZ8706dNh7xFCycnJ8+fPv/POOx944IEVK1b06NEjIyNj2LBhjY2NJ0+evP766zdv3mwwGCilYH3wJBOM8R//+Md+/foNGTIEIKb28nPb2PW2i11g1jDZmJiY8ePHL1y4cOTIkeHh4TIOqTNsh/DxafPlitZut2dkZMiybLFY7r///szMTM3fpaSkzJ8/32q1arQgnxo8x2q1ZmZmCoIQHh5+7733ZmRk8Cdcc801e/fufemll6644opJkyb5fD6uzC422AWWxJEjRxoaGgYMGMBHq/nccx0GXz2+hgMGDGhqaoqPj3/00Ud79erF3zlmzJi9e/f+8pe/HDZs2Pjx48Gl7rBQPMb44MGDXq83IyMDRhv60WeDm6vTSdXY1zvvvJOdnf3Tn/5UfdDgbZIkJScnnzhxglI6bdq08ePH8+fn5OTExsaOHTt23rx5ixYtUkcx292DbHGaLpdr7969vXv39p81WQaXq/3BrkAVsiLcVFKOWxVnFBHnIXba+Plp6wA+GWKXT0YSaXdME74SQubNm/eHP/zhL3/5y913380Y8ykky7LL5TKZTD/72c9KSko+/PBDwGExxrt27ZJlOSEhIT8/n0O0PLGVEFJYWLhhw4YpU6aELh8IHUmSZFmWJEmdc6LO/Wh78Jq8Ef4oIFmWQzNJ1CnRlNIpU6bU1tauXr1aEATKaMuW8LmTmr1bs0YrKio2bNjQs2fPHj16QPgEBgzoIiFk4MCBNptNYzfxiXMOrq2t3bBhQ3p6emxsrCzLHo/H5/PB16ysLK/X+/3337eokM5SOJ59To4awccYr1+/ftGiRW3koLeYdCQFSZMoCM6Z+jmMscrKyvXr1yclJXXv3h1gLpg7pTQzM9Pn8+Xm5mqW67xnfcYVUCv+r7/++tNPPw3N0AsVsuo5+nw+9dz5Z3F+JoR8/PHHjLFf/epXVqv1xIkTbrdbfRZkWaaUrlu3rqamBlyQpqYmSZLcbrfP54uPj09ISNi8eXN1dTUokrO0ms5mWVpbEz7f+vr6v//97+Xl5ZoUg/Z2glQHV5Xzwo88LC9IHs0U2vYY1GJHzbfwQFh5+Hq6BFNVUzZXNeIupU64rcdtyXfffff3v//9r371q4kTJ3o8HkIIBDnhBaU0JSUlNTV11apVEPOUJOnf//63xWJBCJWVlR07dky9c7CUX3zxRb9+/cA+1WwV7A18hCiK3HoNVRKhbKp+FN88QJD5mLn0aQMwYYzFxMQMHTp05cqVlMqC37Shrcm7s5cyp30oa+GjYabl5eWlpaUDBw6MiIgA99zpdNbW1sKCu91uq9VqsVjUgdbT62wDzzl06FBJScmwYcOioqLgOYQQABU3bdqEMR4wYIAmFn3GTE21vGtRWaptkRanD7Fxl8ullrOhy6UWwTwoAqTB92Fb1fuOMS4pKamurr766qtB70L+AjhheXl5CKE+ffpoDvBZasS2qUXJol4Bt9utxsdbE1uauRsMhtbmzhgzGAybNm2qrq5+5JFHIBT/zTffQPIFfwgAAIWFhXa7/YorrhBF0WQyAfJmMBjKysoqKioyMzMjIiL4cdOc2RYtvzaYIZQrWmMMWZYhtae11bt4IBhrblTDuMyBgJNGX7axFJoMSS584FEAR7e0eqc9wi9vL6dOOGK74xIGg6GoqGjevHlXXHHF448/DkzJI2a8dM5oNJrN5sOHDzc0NMTFxeXl5e3atWvWrFkIobS0NPBUOEIN4OCmTZtGjRplMBggaqL+XNjLwsLC+vp6URQzMzPBBdZ8bouOpGZTYV8NBkNjY2NhYaEgCIMGDYIkFthvNa6iTlOGj+vXr9/69eurT1XGxcdrYif4dA47o03B/byzMfRKSkqampr69+8vimJTU5PNZisqKqqsrJw0aRKkgU6cODE0G1sjFhlj+/bta2hoGDp0KGTycAtgw4YNX3311e233z558mSe2K1xFM6YiHXG37aWIwQ8E4rGhMbqOR8Czxw/fvzEiRMJCQmJiYncJGeMVVdXh4eH85gz/GF+fr7b7b766qvBtIdTbTQav/7662+++WbatGnXXXedBmQ7eyS97YSONvgBYl2g0VtMltVgIHx3jhw5UlVVlZKS0q1bNx4pkSSpuro6Pj6+sLDw8ccfFwRh5cqVsiwfOXLklltuuffee9XSWRCE6urqXbt2paeng38GctxsNnu93r/97W82m+3pp5+22+08S1ODd4UuSygztD33UE7gOh6Ys7VluUjFlUrxSQBiEgTh6NGjx44dQwglJSWlpqaGZqm0WEUfagrzXSssLKyrq0tPT4+Pj9dgnqqHNOOWBHc5daI2byVJevvtt48fP/7b3/7WYrGAma9eEeBUp9PZ0NBgMBjAVgoPD3/hhReGDBmCMb7jjjuOHj3Kzy1sW2lp6alTpzIzMzUCHXi3rq7uX//6lyzLSUlJsiwfOHBgzJgxSUlJ6oe0GB/TdEnhmc1bt2799NNP09PTCSHFxcVQ3nXfffeprTBNoT78+ZAhQ3w+X35+wbXdu2OmXKIVcjzOCVxmZ7JA4Gnff/+9xWK5/vrrCSFhYWGMsTVr1gwePBhcqziFAEJsTabDydyxY0e3bt3GjBlDCAFn0ePxrFix4g9/+MMDDzwwf/58kB2wp9wcO2NTNY3tzE0ENVeE2msa404DS2qEkZolBEHwer2vvPLKO++8U1FRkZSU9Je//AUUIeiYTZs2TZ06VRRFrrNlWd60aVP37t1HjhzJ5+5yuVasWPHSSy89+OCDv/3tbyMiIrjcPJugq7pjAhfQ3C7R5CNpkXEVn7do4WqWDt4M+tLpdP71r3999913a2pq0tLS/vjHP06aNAl4uKys7Pvvv7/zzju9Xu/UqVMBDcMYZ2dn33jjjRyz4pq7tLS0rKzs4Ycfhlxh+OgDBw688cYbhw4dWrhw4U033cQLktS1Ta0tS6jNzv+WQ9xcLbUYmg113EMh6LMPhp9PnBgzJjNBFJYtW7Zr167evXtjjIuKijIyMq688soWo4ltsDrveVNZWfn2229DpHPPnj1xcXGlpaX33Xdf9+7d1aaSFvVGXU6d8B0Fdb1q1aqUlJSxY8eqz4DmzFdUVNTU1GRnZ4Phn5aW1qtXL7Ar+yrEDyH4N3v37m1oaOjVq5emtwrszTvvvHPkyBEQdtXV1VOnTkUI3XvvvXCENKn0LbqcaohgzZo1s2fPfvbZZ++44w7G2Mcff/z444//6Ec/uv/++3n8TX2E1E/r2bOnJEnFJcXjrh/v33TSgkmSn59fUlLSNpdjjH0+X1xcnF+68etx2GkLzqt26uvr//e//xFClixZAl2YduzYcfDgweXLl3Psrg1LkEf1GxsbN27ciDFetWrVunXr3G73yZMna2tr09PT58yZM3HiRKhH4fYBt0PVR4VjRxqdzUUGz8cLdUfUWA1/IGRYwQ956pH6iKpNVC6SXnrppa+++uq2227r1q3bpk2bnn766dTU1IEDB0KUzmw2m0wmeDOon6qqqtzcXJ/P9/nnnwM8WF5e3tjYCOJ43LhxfO58jprMXf5zTdsrddjW6XQyxsLCwtTSU6N7OOfzp/Foh4aT+dzVbjpCaO7cud9+++0999wTExOzZs2a//u//0tJScnMzKSU7t69Oy0tzWAwZGVlZWdnh0LWGvbYs2dPfX398ePHFyxY4PF4GhoaSktLY2NjBw4c+OyzzwK3cx2gLlhuEYVT58Vx5A0ic0aFQkNB6mYNnFdh9YCxgT3U8hqW5SIVVwJDioK4cePGDz74YO7cuQMGDCCEPPPMM6tXr77yyivVQk9jSWtUC6w2sF9lZeWDDz6YmJgIVkthYeF9991XX18/Y8YMNQrif7NSz3Z53snennUncMD27Nlz4MCB66+/Hpy1UMsRXufl5QGoYrVaeWQCzgYwCrePgBcrKioaGxshMKDW6qDAli5d+uSTTwLc7/F4cnJysrOz+QND8yw14o8bjJBB9MQTT0ycOPG+++7zer0mkwmO31VXXaVWYOokRbUVCSCew+FEcL9viGNBKd22bdt3330HyTAtuufAYW63u0ePHkOGDLFFhre24LCeR48ePXjwYEZGxogRIyBStXPnztTU1G7dusHzYQU0FWfqIYHOLi0tLSkpGTNmDDxHlmWbzbZw4cLIyMgbbrgBIQQQEF9JvggaK1tdKa1JJYIC1YqKipiYGHPwwmO1IIA/V+NyUAZrsVhMJhO8Dh2/WmwJgvDll1+WlZUtWbIkPT0d3jNnzpwlS5YMGjSorKystLQUDAX1CA8fPnzixIlx48YNGzYMwqFGo3HhwoXx8fHXX389aDKePsR7nWmsEz539c6C+PN6vWvWrDl48GBdXR2l9OGHH05ISABfR617NHMHMplMZrMZMPq2504I+eqrr9xu91dffZWSkoIQevbZZ59//vnVq1f379//5MmTNTU1Y8eOBdRLM3g4LOr2bpTS3Nxcg8EwYcKE3r17u91up9NZWlq6fv3622+/vWfPnl6vV30QNIeiRfmujnNwb0kQhAMHDvTs2XP69OkwfU0rudCJR0REQIVpiyum9pvbK2Cixp8poh999FFUVFRWVhYMMjExccSIEXzufCk0cRSNLuFGw9y5c6HqCyp7+vbta7Va+/XrFxcXpzYcgyDzZXpnYftXxR8/flySpKFDh5pMplArFVSO2+1evXq1wWCYNGkSZ2WNHafmRTDVjUaj3W7XfCJA/CdOnHj11VejoqJ69+6dlpb2+uuvqzNBN2/eHB8fn56eru4m0hrm849//OPw4cP33XcfB77y8/OtVuuIESP4h8qyvGPHjvj4eOhQwqUMqBOr1dpQVx/ID6daTJAQ8uMf//iBBx44IzrEuc1DJQRVLFildVTiLC8vz+FwTJ8+/a677oI3pKam5ufn89YLXGy1NmuQSvCcW2+9ddSoUfCr8ePH9+/f/4YbbkhKSnrmmWfUqGZTU9OWLVuysrJiYmL4UsPTGhsbDQaDOnkUhlFbW3vy5MkVK1asXbv2nXfeSU5O5k4eHwPGuKampqCgwOv1woAhAXr37t2lpaUbNmwAT4ubwxkZGcnJyXwlBUFwOp2HDx9+/vnn09PTfT4fKKTnnntu/vz5DQ0NR48eTUtLg3AC5zTG2LZt22pqau68886rrroK5n7ttdf26dNn8uTJaWlpM2fOVBs9jY2Nu3fvHjRoEESh1cgMtBWAuaux/pUrVx45cuS+++7zeDwPPPDA1q1bP/vsM4DUOFMRQqqrqwsKCrg3iRAyGo2FhYWNjY3ff/89r9WHv8rIyABEl0csvF7vtm3bfvazn6WkpECow2g0/uQnP/n000+dTmd+fn5iYmJUVBS4FC12NlIXnVRXV//vf/8bNWrU3XffzQX65MmTR48ePWvWrDVr1sTGxnKxSAjZt2+fx+MZMmQIj11xANzlcoFPpjYaTpw4sWjRohtvvHHo0KHffffdPffcgzG+++67gV25IyJJ0v79+0+cOMHtQqPRWFVVBaqovLzc5XLB8GRZDg8PHzJkCDSn4Dnl5yzKcEspXix4JDFiMmtqalq+fPngwYOvvfbapKSkF154gath0I7V1dWgYPhc1F0JuNoWRb+js2jRotmzZ3fr1g0qkQ8dOnTs2LG7775bEASw4QghJSUl1VXVV+RcQaninVx+HorYvinhIPehuESd7qXuK4UxLi4uXrNmzfXXXz9mzBiNga/WJZyfwLgzGo1gnKrVjCRJaWlpP/3pT3/xi19Mnjw5ISFh9OjR8+fP79WrlyRJRqOxoqLiiSeeeOGFF9LT07l6C/VOuJj47LPPsrKyevXqBcAXQmjDhg1xcXFJSUnczqqtrZ09e/b//d//QW8DdSkAvMHr9YCSbDHPRx2obCMC2fxCDnzTor2PMd65c6fJZMrOzqaUejwekGX9+/eH8p2GhoYNGzaMHj06JiaG9xIOhZgwxvv27RNFEeYOEk0Uxbi4OLvdvmXLFq71JUkymUx79uz56U9/+uGHH4I6UWv9r7/+OjY2FgLXan/L4XDU1NQ4nc7i4mJNuhE/ZoA7rV692q2UgoLOsFqthYWF8HNgMNhESZKsVmtKSoo67cJgMEyfPj0uLo6HNymlERERWVlZBQUFALFqYhIIod27dwuC0KdPHw4rGQyG7t27Y4z379+vVmCCIOTm5r7wwgsfffQRdABSW50rVqxISkq6+uqrYUjw/oaGhi1bttx+++3QEevJJ5+86aabNm/ePH78eP422E2YIyRBwJPNZnNxcTFYYJDICwOWZRk6F2iiCD/+8Y9TUlK4QIeAotVqzc3Nraio4IZCi+FijZt48uTJ4uJi8Gb4kERR7NGjx1dffXX8+HG+yDCA+fPnJycnDxkyhPvBMKmysjIog42IiFCjnTU1NW+++WZiYuJghWw227fffnvXXXepYS5wpHbt2pWXlwdKAtLS4KaWrVu3QhIKqBOfz5eUlJSZmcldf02o9exyg3HL3gkO9PRl/imLjz32WF5e3tNPPw1W7Jw5c26++WbOIS+//LIsyyNGjAjFJ7kfyT3LxYsXC4Iwfvx4WF4AqyFrjrsvkC5bVVmlqBOJUXQZeijt3GSFMWa1WgFx4kkp/D0gy8ADkGV57ty5JpMJisJaNMzVDGEymVwul7pFHTcEEEKPP/74yJEj165du2XLlsWLF4eFhS1YsAAeW1xcbDQar7jiCvXJ4c1FNJ9VUlJSVlZ2ww03REdHezweo9FYXV29f//+YcOG2Ww2rgny8/MZY4MHDw5N2IUnBwwxxk7LxA4e+uPHj9fU1LSIq6pPNaTQ9OjRA4lYkySoyI8AU9bW1m7atAncLw4LgFkE+ExBQUF5eXl4eHgoMs7ZWhTF+vr67777rnfv3n379oW0Gfjt/v37a2trIyIiuAMBI9+5c2dGRgagSZoQSEVFhRrs5hBQQkJCSkpKXV3d0qVLNaEsbkxQSnv16vXCCy9wDQFmwfvvv79nz54XXnhB47pxH0iNNyYkJKjVFex1//79t2/fHh8fD0KQ2ysw9z179gwaNAj6ynBcrqCgwOfzwdarY+nbtm1LTU0Fqa1JSKmoqAAQTz1ISZJWrlxZXV39+uuvC4IAQN/hw4fVtiosUe/evX/xi1+o2VIQBLvdXlNT8/zzz6t3EGNsNBq5j8WdY+ijo9Y6RqPRZDJt37592LBhPXv2hBPHp682+DQcuGPHDqg34lAY2AQHDx602Wwmk4mrZFEUa2trDx48ePPNN4cyM1TRe71edW40xJyff/75fv36gSfkdDohuKU2QBljJpPpzjvvnDZtmno3KyoqqqqqHnrooX79+vl8PmB72M1A3zxVVqc6E1dtz4Hvq8EbJVkSBbG11Hy/B0wCR2zFihWrV6/esmXLv//972eeeWbo0KEpKSnQFu/AgQOAcKg1PY87qvMP3W53fn5+SkpKRkYGH9vOnTu7d+/OTyIse1FR0eSbpgSUE2Fd2TvhjDJ8+PDIyMj8/Hz41ul0FhYWejye/v37R0dHY4wXLVr08ccfv/766yNHjtRgx631T2WMRUdHA88lJiZy7JUQsnLlyu3btz/zzDPDFGKMPfDAA+B019bWFhcXf/TRRz6f78CBA4QQOP8881LTGYIxdurUKVEU+/fvD3vPGDtx4sTBgwcfe+wxxlh+fr7NZjt+/PiiRYscDsehQ4fMZjOMhwtN6KIY0HmUqkPxNPi2zz77bMOGDTabLTSYoX6Uy+VKTU2dM2eOPTqyjVqfY8eOFRUV5eTkpKam8uiuyWQCcNzhcCxfvnz69OkGg4Ej3ZrwOJzP0tLSvXv3Xn/99d27d+eKnzG2Y8cOr9cLzhn4KwcOHCgtLV26dGl8fPzOnTuh8gDq5EGBgfnJa7JAGPG0b5fLxcGcFvMCNEECOITwTDBWQr2r0GCpOrGYw1BbtmwBI4YnaMHfnjx5cv/+/bfddlt0dLQ6iyY/Px8hFBcXB481GAwlJSUHDhz4z3/+k5iYuGPHjt69e0O/H47ZAjNDqjGfe1hY2K233hobGwuzOHr0qNfrhbyS0Dw9Xm2qDtRBsllrxZtqc17d/Fi9IHl5eXfddRe3EkKD3qEJeNu3bzeZTBAe4HxSXl5eUlISFxdntVohwuR0OgsKCjZt2nT06NG6urqioqJevXrxLj4GgwEylyBmDssCXGq3259++mmv11teXr5s2bLrrrvurrvu0uSCw7cmhdSTApzQZDJBUC3UJG0xrKW2ROHJgECo5240BDICUAjqxRAzGMTSI6X/fPefU6ZMycnJmanQpEmTnnjiiSNHjiQkJOzatWvPnj0HDx50uVwFBQVQqwRLx1lO7Ys7nc7GxsbevXtHRUVxkxRstdjY2MLCQpPJVFdXt3Xr1j179lw79tri0gMGZf26Ptgly3JmZuZtt932+eefv/TSS/Hx8R9//HFERERkZOR333130003LV++fN68eS+//PKsWbM0uHlrjSvgDYmJiREREbW1tXwnwCCaO3cuY+zRRx81m81gSNpsNkgYAw47dOhQTk4OF0OwhcePH6+rq+vfv786ZRNjnJCQYDabITMPzvD7779vMplGjx5dUVGxdu1aCOEWFxcPHToUXBBNtlhdXR1jLCYujoGsVIXicXCVZsyYMXny5DOmCwPI45fUVNbETrichTv1XC7XsGHDINuVG5twTj755JPGxsbBgwdr4lhqtAfO/IEDByA5Ql0aJstyUVERIQTiHB6Pp6yszGw219fX19bWjhs3jgdOIS2Hh83DwsI0MVJ1sWHbnfDVhiSXjzAqda6zGvNsAzLlz4T6dnXv1UCWjijm5uY2NDTk5OQAzMXVIZTTAkgINZ6CINTU1NTW1k6dOpWzDcwdyGazQUULyDueTTB37lweIPnwww/Hjh07cuRInlarDmup5wh8yK//4faH5qsavA3NwAb3CFpOhB600I3gsbGdO3f27dtXbTBB74Campr+/fuDEK+oqGhoaDCZTAUFBZmZmb1791YHjYEfoqKigJPVLAGgoiAIhw4dWrZs2fr165999tn4+Hjebk4TY9AkqoC1xPO7IOc7tPpEDS5VVlYeO3ZswIABoOHA+CssLExLS4uMjOSG5t59eyNt4YKZ8Ov11aF4Igjvvffe7373+5EjR/p8PpfLFR4eHhER0bt3b2h0BEvRq1evfv36AavDpjc1NRUVFfXp08dqtaoDKna7PSwsLCYmxmAweDwes9m8fv36Xbt2zZ4922q1fvnll+PGjbNarfv27YuOju7Tpx80V1FQt64LdvHkfUEQXnrppZMnT86ePXvKlClms/mOO+5ACL3yyis///nPGxoa3njjjSlTpnAgvrUUda5jQEsNGjQI1nT06NEcojEajTk5OQMGDOjWrRsELVevXn3ixIl58+ZRSrt37w5ZyPfccw/kZfFw5XPPPbdx48bFixePGjWKR7oopX369MnKytq6deutt97qcrk2btx45MiRmJiY1NTU7du3JycnJyUlQf7YnXfeOWrUKF65ynMZCwoKGGODBw4MSH+CNXoRrF0weM+GlBu8aSB2wk4DuwBU+e9//+v1esePH6/OzHG73RUVFatXr/7zn//8xhtvWCwWfjNSqLaGiP3atWslSRoyZAiPWIDOBvAEMou++eabo0ePPvXUUzU1NREREXfffTf0tqqqqoK6cVmWrVZrQUFBVFSU3W6HaLDJZBo+fDgY16GFFGoeCK0Y4DIUZqfudtfGFdmaTDP+qPHjxxuNRsBG4Idms1mSpC+//NJoNGZnZ6uTmAkh3bt3h7xeQRCWLVtWV1c3c+bMsrKyiIiImTNnQtbNqVOnCgoKuD+Un59fWVkJjhpjzGKxZGVlgScK8Zj333+/rq7u7bfftlgs6itk+PTViYh8WdTTb3G+bUfjGhoaoJZI/YmabFpekAs6ID8/f+fOnVOmTImPj1ene3Xv3h2ybGJiYqqqqj788MNJkyZlZWUxxiZMmHDddddBCG337t2VlZWgP44fP75v377Vq1fb7XY4MmlpaVCuwRhLTU195JFHMjIy5s2bJ0nSjTfeCJ6upkI+NFVMXdLYmjHKT70kSS+//PJHH3303nvvTZw4EfT0F1988dRTT/3kJz958cUXARUsLCycdvu04ZlZ8p/GKTacWsAFHpiW1nPmzB+NGTPGoNCBAwc+/PDDu+++G8wOwMCvvfbaCRMmcB9dFMXPP/989uzZTz75JAC2vLGh0WicOHHiypUr6+vrIyMjc3NzN2/eLAjCwIEDa2trrVZr3759IyMjjUbj8OHDJ93gf+aRksMYsy6rTjgvAoKUkJCwePHit956a9GiRdHR0XC6/vOf/4wePfqXv/xlVFQUIEU9e/a02+0cENAg+5r7qcLCwtLS0vbu3asWSTab7fnnn1+5cuWCBQsAoq2srJw3b97gwYOBI/Pz851OJ7SxUnsncLnp119/feWVV6rLpiwWy8svv/zWW2+9+uqrRqMxNTX11Vdf/c1vfjN//vwBAwbcfPPNlFLA8SBeralcQQjt3bs3Li4Ojoqm2ogF4x/nnMUYwjYE+x/y2muv5eXl/fe//zWZTO+///6qVasAnqaU1tfXFxQU7Ny584Ybbrjmmms4aKtJw4NWev/4xz927969cuVKo9H42muvFRYWzpw5MyoqChb/mWeeKS8v/+qrr2prawVBmD59OqV0y5Ytdrs9Ojoa3lNdXb169WrQQHa7vaCgwGq1er1ecAiioqL69+8PRlmoZR0aqNRAFqHlxOdafw61R2VlZRkZGTyNShCEurq6hQsX5ubmrl271mQy/fa3vx03btysWbN4qeZPfvKTY8eOLV26FOCp6dOnM8Z27doVFxcXHh4OYoInCEBTlry8vKioqMbGRo/HI0lSXFxcv379wsLCQHDs2rVr6dKlL7/8cr9+/UK7JWqqFEN/cpYTVy8miF1Jkvr166dJCQmt+OOJRp999llubq7H49m7d+/DDz88Y8aMMWPGAOump6e/8sorn3zyySuvvCKK4uDBgzMyMhoaGsrLy2+88UbuRuzcubOwsBAE7smTJ4uKiqDND7TCGz9+PNQIgzVjsVhuueWW11577aWXXho1ahTky7W9JqF80narG/gsl8ulzvCGtHVNvpnfbaJUADQJq84gVdpi+nx33nmXPSx84cKFEPqtrKycMGHCbbfdBgiqx+MpLi6+/fbbQXFyGw5+tX79+vvvv5/nNILYeeyxx5xO55/+9CeovXvooYfCwsK+/PLLw4cPT5gwITIyEgTmhAkTFGdaIOCdsHNgiU5WFa9mX5/PZ7fbf/azn917771weUZkZGRhYeGoUaMAH6yurv7mm29+9KMfqRN/W3smVx6TJ09+8803q6qqeGaqLMupqanTp0/fv38/ZCIOHDgwLCyMl1Zt2bIlKSkpOTnZ7XZ7PJ7IyEgAN1577bUHH3xw06ZNTqcTrgziCbXDhw//wx/+kJeXl5CQkJmZSQj53e9+t3v37szMzJiYGIzx999/37dv3/T0dK9CYWFhakbcv3//mDFjjFarjzGDxoQMZmeda1U8YrJGrzCEBYKHDRvWr1+/Rx55hBBSX1+vDqsSQkwmk9Fo7NGjB9iSLRaQg6jNysrq27fvww8/DBLWYrEAegMrnJSUtHDhwl27drlcrqFDh8JR3717N/QHa2hoEEWxT58+v/nNb2ABDQbDggULEhISpkyZwm0FnksN6Dm0BNW0t2vRQ20OkEoSN1rbtsRDxRAhxOFw7Ny5c9SoUeoVAI8kMzPziSeewBjX1dXZbDaYOxhG6enpCxcu3L17tyRJOTk5ZrMZ7Ikrr7wSEG1BEDIyMn71q1/xIf3rX/9KS0sbN24cj2MZDAbIJigrK1u+fDl0Jq6urvb5fOBVt+Zj8TPF597iDraGPHOs7OTJk06nMykpKbRdoKZUGz4iISFh3LhxU6dOBZugvr4eEjU53z766KOTJk0qLi5OTU3t06cPIWT79u2U0pycHAADw8PDIbWat7T44osvwEaBZQEv8PPPP1+6dOm8efN69+7t8/nS0tJWrVpVVVUFoccWk7LUfUu9Xq+6EjCUZ9QuviAIL7zwwsyZM0GtAoR4yy239O/fHwp04OxkZGR8uWJFnDXyBXEtkmlo0pffUzdZJt80ee++vdXV1RjjqVOnQjMO+JRdu3Y1NjZmZ2eDqREZGWkwGBhjt9122xVXXLFq1ary8vKUlBS1JoiKivr1r38Na5idnW21Wh977LEdO3bYbDZI7iouLnY4HFBL63A7fAE8kF1W/R/bWZ2or/+DQFyiQvCet99+e8OGDdDsYdGiRb17946Li+MpGW337YHnT5w48a9//evXX399//33AxvxzvC8VgBgYp6EB4FiaDkFOBW4vdHR0bGxsSkpKbySjuMMlNLY2Njx48dzeDcqKuraa68F8N1oNG7bti0rK4sQsmHDhrS0tL59+8JkCSFFRUUVFRVPPvkkZoyiEG/09HakZx+RQgwHwS7FBKN++cIQhrvF2iYIeKgr0jUN9YxGI8yuRQAdNITRaLzyyit5SMPhcJSUlMyaNQtjvGHDhlGjRtlsNrDNAS4HuAMQanXODOiViIgIk8lkt9vVaOcZGWDIkCExMTFn3yBLXUiMMS4sLAQ/lWdtIYSsVmuLc+csAdgdJNeCvKisrDxw4MCMGTNcLte6detGjx5tt9t5cSUwpNfrhZ+A0IRld7vdy5cvHz9+fHZ2tsfj2bx5c1hYGNw9xXHd1iyq4cOHOxyOc+oaos5XzM/PF0URRJh6ZdSfqL4TrJdCrRXMw4zgKk+EUFNTk8Vi2bx5s8ViSUlJycvLczqdI0aM4KY3XMnFC4B4SSN4ctCFAdDm0tLS5ORkgBDV2ZIt7qzdbp88eTL3j88olyCdB97PgT6j0Thw4EBNj4N+vfv4t/uEfBowEKw7UR5FRYMICW/qcBecjl27dhkMhsGDBx84cKCsrOzqq68G0wTWJzw8nMei+HrCp48cOVKdTAEnDtLl9+/fTwhJT0/fW7i3tqk2mSSGtsVszaHvlGCX5jXvRMJNZp/P969//ctqte7cufPUqVMPPfSQpgtpG08GKR8TE/PEE0+sWbPm9ttv53nlfBfVhWycgQD13r17t9frhRgsT2vJzc2FygyeN8k5j8eiuSbjCgNjDMDFnj17fD5fcnKy2vBfvHjxoEGDsrKyJI9XNBnbWKazd1EZuLXB86EUMGKL0XT2OI+mWg368vI25hoNp/EIeQNaHgQGGREZGel0Ordt2xYeHg51vJomZuqKLQ65NDY2Hj58eOPGjSUlJWvWrMnKyurZsyfEt1pMxVFD/CNGjIADdpZ9/TSdJXNzc5OTk6ExqHp2PIdNzbp8LuoaTIh8QKFcbW3t9u3b4+LiwFHW3DmvLvzmF4f85je/eeutt7KysqBvfENDw3vvvddasylNUP3aa6/VtHI4m/6h/Mn79u0LDw8H10rTw7jF53D+1yhX9Rx51BAGaTabLRZLYWHh0aNHhw8fzu9C5sNWLwtfrmnTpkFRTm1t7XfffVdeXv7rX/+ao4it9YyA59jtdmh6xAGlFgFzTdaJZgV4Z0z17HxUNhCl4aYqWgkaJTgkAq1/1KklfGyiKEZERBQUFBw5ciQrK8toNPJir9zcXFmWeZmUxjCCdEeuhtVSyGAwhIeHFxQUlB0tGz5yOGqQqaxwryTL9LQsjIt638wlALs0Uom3BxgxYsTixYtffPFFq9W6fPnyhIQEtbHcYsG2uhMc7P2MGTP27t37xRdfzJgxA+wddXa5RioRQp5//vmSkhK4AQkwH3hmXl6ex+MZNGiQuvWW+i6pUOHIOXLOnDmHDh3CGF933XWABYOg2bhx47Fjx+bPn6+wLMKhjXXO2zNlwboTmVrM5kN7S15Z8IaBCGd5vyzXCvX19cOHD581a5YgCGFhYZCOdTb3X6kb1UBJ4Isvvnjq1KmIiIihQ4eqfQiAGeGmEHWPA54qevz48V69ekGRF9xXz/OdNOKgtda552qhw9cjR46MHDlSbZlyiFJjw4YGt/n5pJTGx8fPnz+/qqoqMTExPT1dXQQKI4QLZtS2CBQ2Dx48+Be/+AX8yufzde/eXd3uPnRGbbRhPhtdwmOZlNKKigrYptawxFB4WbMULa4PPByQnJkzZ2ZnZzc1NV111VW8rFXdHNbj8agfDsPLzs622Wz5+flFRUV1dXULFiy45ppr2hCImrB8a8KnxYwMDpxomuuEomQCws2Ne7F2ccCo0/gW3HRjjN16662pqakOh2PEiBHQfAF+VVtbW1BQcN1116GQe1N4Cr7afVSXy0yYMAGwnBEjruzerVupfBAT//vDIyJMZpPmXHR676S1Q67e+Pvuuw/idbNmzcrJyVF3tG6tYbtmgSDx/9FHH12wYEFubi4UgWvqeNVSAAriwG1XW+JwSdT48eOhCaCmO1CLh039EWooACwIURSPHTu2cuXKBx54IDExUZIkQRRp+zfWYeBxL3z3n3977fXze8Q777zTo0cPs9m8du1ag8HQs2dPnjzahqgKbcDFSzg1AhpeDBgwIDIyUtP1D3yaSZMmqeGvFnNb22azc8pcB+NAkqRx48bBmFtLBDjj87ny5iiH2hHnSzRgwADIB1PDR0ajkfe/CZX7XHy0NuXzviefEOJyucaOHQu3BLXdiOGM/N/iAYefQBMETaIz54eIiIhhw4ZBLob6CdCcqlevXvX19Xa7nRs3Z2kraC7vaUP3nBMLYTXCFUyz9rg9Ku8q8IwWuSgyMnLcuHF8gjyb7uTJk4MHD4ZEas15aYP9YNMjIiLGjh0bqHyhDCMsiMKJE8eLvjx4vOxYx1w1fQnqTlo0lmG2MTExL774onqVQ2OAZ+R1SZJ69ep177335uXlZWZmQiGV2kDQtJ9T1wmq1QYg5hpkv21LX42nq0cLs9u1a9f48eOvuuoqdcvbdibF9RYIuWb0NVdsXKv4P+ycJIskSX379h02bNjKlSs/++yzVatW/fKXv+Rt2zVGX9u+Tmg1tRpphMiT+thoUHiOMJxRc5xl4L1txSMIwsSJE1sEys4RdWzmXg1OyF+r564RnZpaEE3lXRt3FbfYXOuMc4fFh2sL1DZva33jz/Ly8xYTz9TiTHMtDdwFkpSUpAbHNLVTEBXjrbpaEwUalmh7WS7kEhpVLqX/VXJKjyFZWWwVQ62A1Wokg0MgnENgnHDltiYp5iwHyZ8pS7JgEBhlFrN567YdH2/9pF9Cut1uv0wC8u3pnYQ2/9G0GdeUF2gyN9ooiVfD6JTSgQMHQj6Juitni+BAa9qCx0LUyFgbtrl679VnkpdAT5w4Ud3xgjAmo/auM2IYEex0ua6/9Y7h143ClJ2ruSpJUlhYmNFo/P3vf//iiy9CMxheLdxaQLhFKRwKO6h77XF8srXjp7boW8RSzuakneX1ujw5Sl2X05qqa1sqqWetUQaclzhrtegntag/WpzmGSX7Weo/dcCgDUjkbBLG2u70pfk2VMKqMTRNGxU1WKQBA85D553TjNr0TvzmG0LomiuvGho9YPkXK+NIRGgZaej9Ja1d76a5Ubs1ryI0cKBS0gQj/xePy3PtzWOeevEpERnC7DaNhOz0YFcbab5t3+TcBn+0KMJ45wa1XAhloNags1Cea7tPnOYu0hZ1DOD+XGBBiQlB7d3yEzP4Rxm12WwiJufnREKSCe8aoK6eOXskpDWV36LQacPY1Jj25yHgztLKUVuy55Rfd5ZXkbdYiRmKh7Qol1tDetvFwtO8aK/knzbOWoufG/q6Df+jbY+tA2VlIKmSsGCLoDZ9prY3Uf2GtuGp1i45VbIDAqdVZtRkNEdHRFPfaW0OLi3kdQHqhFImy0iWWxs8DnnR+qad4VeYseY0WYYEpYgP8UJBxgJvOONzznsALct2RZdgzJSG0dCXgx8Fyhhu50sX/GYTRUxmTGISZeRc75vDoBoxlqhy1wVVnc9g823cxmKw84z4nOFX7II/4lw/mp37x7ELGwO7aM8/7wd2QPUbu7A16ZhBhgo2Ron/lFH1yferAqWy0f9rOdgB6Ezja+0NzbfNt/4E9a+a5R9l1P+PYcH/6YrHA70AOyXYpczQo+S9GY1MEJggdHxWGtj+7NIwm1bZqC/b5OMJqBGbVVmr2gt0U2igYRBFVA7DgoH4/3+x5qOTTj9wwv7DFYYNiMqINcO2cpNflotWAxHIpRucgqUbCQyGECIL+BJmBl+gOlHkTVoM2jgEHS4T8vYiSUKXaHEvE9GHW3otUIYIQXUNKDUS1Y1Ud+46Lw5S/ly0IUvM177KMY7dHkQJ1mW/TjpdBGuVURELS6VKZonpLQb6eGKRRGTYBAupzj1FzMKlwpQg+dt1tNHez2KKMkKY9jxSyS8LdRJIeah1InQS/WQhQs8jZEaIXq6MQbk+vxTkRSgRIQGhOujWcP4sCNwie5Dk3lu1ccqxZUjXJZ1EMuk71Vk3zhKHJFcJ9QVwJ4l567z1pY4NT3/vdciYBI28CxFPEGM/Fw5hmCEfCkuzeGp9slO6tMrjQtVJYNx9E9GKoWhMItpAEDJcVlxwOv4kKxNkwZ93MPlQnyhkM6ImLzIIF7Lr8IfXm2JX23ogczzCRoSJfuQvdy3i/wpNztmlYUCdLmQHzbGIkZmGaCVAi4mJxAyINEeZvA6f3CTjC8EbguzhP9YCYRILRGLPUglJzJZsFYyCLdGK0eV1azxml98dLO2oW3xMMmJRPx06XSIHVckbEfSF0Kk1zeLB2NRlpnOuYBe7WDV67WsVyjIWxeoP/sQkHDvzOaXPFbm0yMOFND+43Jddp2YLhkHTWUKIq2hn1cevJzz1JyEmDlEZE+FyqFvW6XxOLmLtk/bDmht/Ubfr+J+eiJ70Y+sVV1GZksD9eLjDpMqlVycdnvd9bkSVtF0aTIdt2LDM0jcHE4IliV0GI79AsEuXRJ1AnVCGCSYKB3r35bm2r0Dia9BMUN/ETqxREG4XrJIySjBRYC7sqzrZuGZR9KT7Mfb/SFFa51V0eTlxFOlidkQgvYoQ/8G2x8vWCMYY1c+wTh3JgXDXCCMsri9lEla3EtRJZw+on/O4WbdMSUlKZl1FQHXBiK6CCvl3jDiOeUv3KsofIR0p0qnDvGQ4WrLLV7pFbmj0syKj+rLoFHBhg0oFn9jnPVmGlOJEXZ1cxr6p0vWTYeaqqQi0RggW1eukUwdIDD8XEuJpQFKTAwf6NjCdA3VSiV4seZGvrjZ4kUpXYA/SFZW/0qgRM8mSKHkp4rpEj2br1CGaBCoSqDnMRxDxuv2vFZ7Ua1B0aiZB9FqQ7FXusoPQWucXUKSr2YTBKwsV81CQTpWgVno46qTTxeBAHGyKQ4w2uQpRVyM3P3X209mDx06IyYrciNZUKJXtqGsIqK4Vig+6J1i58JJKDk9DBfN5lc6GZ77RRCedLpwDOYdhQt0ViDa5FQkCCkVnvx86ezTHTgyi14kkhwPUjObWd12dXD6bxsAaRNYo6kTM6+GZmvp51unim6DBCmebXfYhuamOdyfS2U8nFLwGCZvMSETUWcsYDQTbdLDrMjvIDK7ADdxzZYuiTYi6HKFXdemk00XiQC4UiNlOBEQbawIXaepYq84eoC1AFhnM2IJkRyW/P78LCKiuWHcSNA+JLULEiDU5oO+mzs06dQAHNsdILGGGCMTcjWd/u4dOXZ49uHeCjCbBFk68jkDqaZcQUKTrHWau4g1mq2hA1O3mpQD6kdapIwwauCHcYDLYEfG4ITiPdfbT2UN1G6ZAsNEWTWQfk2RIH+oCAop0PagB2hUo6KTVgBB1N/FsCh1r0KkDODCAaYhGowkxt5OxQOqwzn46ezT7r4yJ5nDsbaI+LzSF6gIc0gW9E36jMzaaBIKo23Va/xWddLrIHBhosiIQwYaYp4liGujIpC+Qzh5BjcIQFixWLLmpz6cwDNPVyeVoADRf2S8YMEHU4+H34+pog04dwIFB2UGIAVGfhzEMV/Dr7KeT+rITIpgQ9SKlQa0Odl2mBgAKbgsRBIIR8nma78fVSaeO4UDFFBUFhHw+rN/JqJOaPbjBIYpY9lHJhzmioquTy0v3q/8jCMreybp3olNHcyBCDBNCEJJlXp6ms59OTBWQJ6IBUTmQPcSQ7p1cfrpf/Q2l/p1iOHDbsw5e69SBHAgAOSN+SUH0TBCdguzRfK+JLFMSvCqeIT12crnahrAtRiPC/OJ4/Szr1KHeCSaEiYp3Eszb0UknyMgI5P6ZjER2B6RVl2CQLuqdKP+RKEPhiIiiHjvRqeO9E8qoZE3BliiMqB470SnUO/ERMwrvi5S7cBjWyxgv28MMef9uh8eJJLONx0500qnD1AmRJVZT5jUaGRH1m050alYnwexTUl/tkSVmMiNI/tPVyWXnS6q6cvq9ExPCgsC9E/1M69QBHAiMxiiTBSQaTIT3atJXR2cPVaGrhAkx2QnCHA7VQ/GXm/aHi2iUnTMaRYZEuFdVj4Xq1FEcCIxGBREbiCj5MMLkdMdFpx80e8BVfkpw1yB7/OIqaIJ0dg4Ru542UfJplNeN9XINwmYTUlUD6KRTB9igfo6TfHIFRWKgN72+KDqBgKKUEuJ3WWlNOSMObDQz1EV6JnS1nl3qO2io2yUzJBjM/Moa/Uzr1CEcqHzj80pNCJmsmCHaXFyr0w+dPQJNVjCWmhqoaCWiyDlDB7suL82vujsAUY/LRxEyW/QWkDp1GAfyr8zn8bkQNlrhJkad/XQKyiZFo1Aqu+uoaCYG0a9cmN6z6/JT/uoXclOjjyFsNOt8rFOHcWBzE3JPk8eBkMGMESJdIw9Up3YRUJDE5fVJTfUyMXBnpQtMkHSx7YL/BFq6NtUjARGzlYNdOul00Tkw2M6PupuoG2GLHelltDqpBZRiWjDJIzUhbIpgjBHURbq6kS7mTHLl71f4rnpiVu5kDv5Kv6xbp4vOgUFLk7ka/afLatfvitfpdAEFzqsb+xCxRQazh1gXEFCks3uOzf31+FZgRhWUEkku0YKwwci3UL+sW6eLx37NDf7gDZ4mgxERky2gZXT2+8FzSKDhk2LvUreLMCRYwiCkgvS74i+1psfN1y+qrjmBK0+ozATJZ7BZkWjgdoF+nnW6eOzn/4qaUz6Jx20IQ1jJBGEYncaiOv0wOEQVfkfBDKFA6IR53SJBgtWm1j26Orlkmj9whqnMKIXDHCgWU+BrKvmw1CQYwzARNPaCTjq1CwcGgqhU1ho0Cslup98xNhg4T+rs94Nij4BfotxQoBY+PLRmYAgbTGrdo4Ndl2av+Or7tQUh6vskFIsRMcmHfRISDOoYFz/wLSIVOul0luzXzEUYYyJwLsIIUeVQMYSYx4MFBhXxDDdbqTr7/RDYo9nCEE5jj+bd93r8rEGIxpXp1BxCOqPaR82GHvOeKJca6k/bSGUbmddHvU4iGlosSFZfHd9sZuqnWqdzYz9EnQ7XsaP8ir1mCxQh6mrCglZAhMqO03KLdQ7sKuwR0CWy3FR6UA2H8n2XXR4kIxLETjRb33xDeafyXTqfOsGQPAO5EJjUvDO34bM3APUKbCT2/49SGTGfcm0RbtEJBe2hxr51aFuns2Q/KEPDGLu2rav67QzZ5eJOM1GsGoYQlX2CjAgLETf8q/KEZjNWD650OfbwnjpV9es73PvzlF2mgQKGQIdQH6ba7jtqpySgmbgi6Qzs0Sm9ExY8ug1r/3P4g/eOL/69p+I4xiRwPpnSU03y+ozxyBiuwA+n7XezMejz+mqqGKWaVsQ66dSW+QmagBBfVeXhv79Q/vX39Ss/wYQgKjPFOaaKryIxkUYiuM1C7ZE08xkh3toq6nEHzSA9uNJV2ENp6cgoO/7uS2Vf7jr1wWt+IRP4aWD/JUmi4SjYHfQ09uBf5cZ62VHPgs6O7p1clA3jdaS05rhXQo6jTTX/fgNu5g72f8TY42b1J3wmy2neSVDtwyWNDZu+bdz6LZgS+knQ6ZzQDD/buJ2+6kIPQdX//p23qhIJAqOUKQ2tGWaortrHkAyJhUEDM+AHM8YolapPVS/5p1/fKOyHdV3SddiDgtiRTx3y2FDt2kWObf8jgoBkye+/KvtMHPVSE5KDPTvU7MEYI4xRr7dq6XtyZUVA03QG9uiEYJcS51Tim4yERTI3okZUsfRl34ljSBAwDegTSpngOcIajsvN/dXUsKYg19U0/u+rsEFXBPoeBOFvnXQ6I/tBfocYFk6xgZhR1bYDjvXLlOt8GWaBrGBEm4RaJPu8akETNGn8zk3Nkn+aE1KI2YqCqYk62NVF2AMCuARjkx0z5KxGNUv+zrw+hglnAypLohNRlyOUPZAkIUFo3LyGNdQak3sCSsZ0sOuibFgQncQYG9MHhfVABCPvKVTznw+VzZADi+5ze1kYDk9oBq+Df0iUGuWGdV8So8mYnIZkOZCcox9mnc6C/XAwQx2HR4ZnXC8yhEyocvk7stPpd1CUDl2EUcnto1GBu6VPR9VlTIjnWGnjns3h19zgZzyBaCL8OnV29gC71dr/SrMVETOq2/SZq2gP8fuvMgokCrt8VgZlSaexh58fBOrx1H6zxD70KiwaAtiJDnZdFHdSWXeqeJSGlF5xOTegJuRm6NSqd2RHI/MrBv/qy5RRc5Lo9XDwmidGMEKY13Nq2T8jxtzkNxSRHgjV6RzYrxndJiRmzBQjxtSGKjdud+d9z5SwCiP+30qiRSSIeF3Nf6v8w5RhTKo/W2BO6U3CI8FpRnoySNdiD6XOHYfljLUnYCSguoOo7tsloBio0jbBxwgxIcFzGnsoMJffc23a+Z375FHLwGFMZQfr3slF0P+Q7K9YeaLFQnoN9DQi0YZ8Rw45t23w/1yWQXmIjUXIXcc0no3sd18q//OR5DxpG3Gtf/8EUZ3Gp5NOZ2Q/zjDmjCw3RkxGooBq13yOsXKiFGEiuGpRHaLstLInSmUkip6KExWr3om8bjJmiAXLV3T260rsAcVwlrS+cmQPnxsZolH9xuVSTTURxEDsRPLgGkRBWKmBMr9kY+UfvmYfNEQIj2KSD2PSWTxX0in9SWVlCfMvsWgOoz7/PBorUN2mr5sBK49bciMkCM2hEyXkzgimHnf9klfjbnwIE8Ir6pFuG+p0LuwHnbkEi41hhCUkCajqfwvl2ho/y1GGKJUcDUhAWHUlkl8JKfzW+OW/rNGp1oEjeYqwzn5djD1AAQhmCxZtzI2QEVXn7vcU70SMMqqAXc4GBXTHas+GUMow8ezN9RWtirpxplLJQDpRXI100j3j+XZCZJxgRogiwYLc+Wt8Vaeg56PSfMU/QayqLEOyTIjQsG5FU3l+9NT7EUOYkOYrgXXS6ewADb+fAS5IWKTBrqgWA5IrJef2dUrSll9pyFRWilCayxj9QkEQ5Mb6E5++Gjt1JrFYmCxzpF1f2K7EHkwJuyNCxIh4gjEliFDU+P1/eSEckyT1nePAHlSRRScW/tXWf7yl9wAkyygooDrF3DulOoHyQ6ossSmlt7UbYoo6qdl3wHmoCDSE/6BK/rMMSCVSTjkTiCTJNV9/EDl2liE2niGqJOPosLVO52qAKmAXZjgyJix1CGEIicjrRKd2bAxeuMeQ16vE5Zo7Dfs9FULqvv2PYIiwjZqiQCIBvtPZr8uxB4IUUluvgQajwgAiOrljA/V5iSD6LVvJx2TEgs6rnwGUYibXwaKmwu+ibnoIQ8Fjp0r568Q9uwTl4JqSetqTw1ET8gqo0Yvc+ZsChp4sMb/6p4HMLv9uyYQQz/Z1Fd8sj771x4ox0EWuGdCpg9kvkFNOKTGaIvsPw16EJNRkQk37t1NHAyEipjKTvVBQqwCzitzAhHk9xxbMNeeMN6f0QJKk7k+qU1diD14sEt4/22RC2INcFuQs3+U9kI8EZdMlH6L8fkYEb8YYV334V48gho29gWEMikfv2XWR9T84G0pc3ZiUKiYMcTsRi0sNp8iTu0Z2OqGTQSBlhl9ZQwQqs9pP/tItO9M6YDhSaX7dNtTp3OxPhKj/HyMCsQy8wuPA3qhwSxgSSjY2FRf6xQmlft83yH6KxSpjjBtWf2GoKIu76V64oFEvN+mq7BHwXxEyDRjqM2CPCYnRsdYq1LRzY7ANB2IkAHxRzJAsI0FwHy1t+vbd5Ck/Fo0mdadq3Tu5mOofB4j5zUOjkHGVEIOw5JHDUU3+Gqm2EuxB5dAT3gAHE+wuzjv6xX+jb39WMJkgxUuHrXU6Pw4kwe5xxj5DcU8kYMxMlsZK1LQ/L2CmMIxowDtRhAJBsnzs/ZfE/jlhQ3IopUhxTXQO7JLsgTGmxP/VnNqXpA0RDX7r1YNRbe73QXiT4eBVWkS5KAdjXL38H421LPrmHyvJHp0vrEs6qfoHpa00R2Jh/bLsBkSOn3RbwxurkK94N+T3K04nJcE0cIRwzadvxaYj08hJgfaPzcaETjqdowEKADnGYrek6J4DjOX1XsnlNKOmwk2YUSwIjFGZBZs4yTImpG7LOlK8N3bqQ9hgxJ3Q9tTp3AQUAnvXFDVwuLkO09rKpgjkKNkq11QhaOmlfIUWcEgQ3ZUVTStfjb/pDiEhSYHLOl9Yt7PGTiATAixEc0KyYEEEI6G2QbCipqOHAwCCX+XLCnzNECa+E8dOfv6uecxD1sRkKktQwKyTTufNgZBiKIZF2OK6U4ZMTkQo8jU4fA11xGAMVBEE00apLNd/8bbTzCLG3wqVz4GiWt016arsoeAnjDF7YjJjyEgRcSBGTK6jBwP7TgMdQiGly73p64ptnsibHwzUIqHOF9btrInCqBmSxqaUXsiMZAmJCAn1qCZ/K9xag0Sk7GYgMlb12QKzFUVOfVCZNtZha50ulAOV6J1gs4nxaZIDYRGZJFRfuNZbeYoQjAURM4RkRakYTU379jRsWJp4zxwSEaUE/vSUwq7OHsHe0caefSUvohgZGfIcOdhYsl8pdxP9YgjYw2CQPZ5T//5L/JQc0+ArNVdH6+qkA9Q/gF1Kq5XoOBQxAMkIYUQJcuStYpIPC6JyKV7g6l+5tqpqzacke0rY4OF+15IQpsPWOl0YByq5gQxjRuLTA/mDZuQ62OgtLQawggYThbEsO9Yvq/exyMn3Y4YIQ3ojhi7PHoHiJIzF+DRqxn5v1Ih81chVsFW5nYAgCTGsFMcJonfXxsq8feGTHzVYbUjpS90Z2YN0WvUf0N6EMSyK1qR+ojIVKiC5us53rBRbw4gdYdlLqQ9hXLXlf6S6KOH6W4mSMYx4pzbdNtTpfDkQB0IoyJzYyxSOsYxkgmQ38h3azRAmUYkCRkzyIoSaaqur1i1LGzfNmNCDIUj50tmvi7MHtxgMsUm2OISZUgYnIN+R3RgjEtONmBHzSZAnfOLrf3fr28OeNSJgl8D/Oxt7dGJ1Al9lBc+yJKaKRIErBWT0Ic/BvVgQZFsvZulGEGaS5Fj9YZMH2cfcqFQBtHyhpk46nSsTQi2tGNPNFIWYcs2eyYy8pfslj5tZ7DQKwivUs31dzaZ8y+jJgsmElZRCvX9wl+cNFFQJxGKxJGT4fRGGRCNCNYekmmpssshRCCkNp72lB6pX/4v0ucKc1hcyhoO1KEhXJx0HNUA0HmFs6dlXFJXIFkE+CTmKC0SLXXQf8nidxBbWuDevac0XSVMfEmK6Q3c2vUuSTu3CgeBhGGLjzbFRzKskFFpQ48G9tKnJYmC+RkSNZoxJ5Ud/Ssxi1ivG46Bfo3ew/iGwBxgNxGqzJvdBPmXzLchZccpTfsRitdF67FVCaFVfLLQ5WOwN98Dd0J2XPTq9dwIQozk906s0JSAM+bzIcWgviYgymkys4QjD2LdrQ/0BZJ9wD1auOWvuu6eTThfGhFD8ZE7sQWJ7SG5EKGICajycx3xeU2w3dAIhIsp1VXXf7zTkPGBKTKRUpkHW1Rew6/MGY4zKxGQy9ezra/JLW0yQ8yTynDpu6J4ouhDxejHGjZuW+rpb7VeO49hJJxVQpHO7k3DxCcbG1L4o3H+YEUaShDzlexCSWNxQ0Z4mN9TXb18ff8tQY59BDGGmqhjQmV6nC2Q/hjClMrFaxcSBSFZSvQhyVyBaU45jUmgMNprNNd9+GZaMwq+6CROMaPPV1Poa/hCYBLba2HMQFZRsL4J8NUgqLyL2SCkKGSJjmg4WyTVlcRMeJTY7oxRx9tC9kw4mihBROhqIUXFR/UaLDFGCBCsSKgt8VZWWbsnEdbJp6/oTX3xlHX2zMSqaUYkFb0zTeV2nC2c/TDBW0rTC+g4125V6AwFZGPIeyDfEJRgY8h45WPX5296I5LAR1/qZTiCd6OpvnS6EWDB2whg2pfYLT8HYhyhGpnAkH9wjG62mcERrTtWt+Ki2xBUx8Xa4s7xTs0fn9k5w8BYTLIrhA3Owci03wcjViDxlRxBqclW4yt/+uSUahQ0dq2gRFCyQ121DndqB/XCQoaz9h5gjMKPK3Scm1HCwBHmcFKOTi1/z5G+LGjHFEBGp9PUhAVhc58AuzyG8HRRihqRUW89M6laCtmZUd/QIq6vGGNVs/rL685cih/a29h2s9ITq3OzRidWJ0hZJicYruf+W/sM9PkR8SLIiF0XufTt8xCLVo9rtRbasLNvAYUqXJL1Jl07tyn5KBQHG2Np3sBxhp01IRsgXhZz783wOh4RQ496dXgnFTJyu3JWEsOqqHp26uDphwducqWyIjjOkDJBc2G9tRCPn8SO+8sOSEbmPldftR90m3YfNZhq8yq/zskfn9k4CxT6KmWjql23ujmQRkfAoWo2qvvm4YV8uCUeCBdlH3iaEhSFZRkoehW4Y6tRu7Be4jZUKUbG23tcQhFhUhOBGDbtXnFq1BNv8csEyYIA1e5SCmxP93s8fkMEB/ivG0HnaOuAq0YioHREBeUqPVix/T3IyIiBTDxR21Y3KLU0Md3L2IJ39SEPrJMaYGNstcuhEwYhIbS0yIOe+3a6DhxFCtu4ociy/qkhP0NSpXTVKoFSNIsQis68xxCHBXU/dSHKi+s1rkBlhF+o+6T5iNEIvBp39foDsAUkb1n6DremxREL/z963xzVxpf0/MwGCCOGOCkGSGWzVtRVQQVsv7GoVC3gB2+1q6637tlXx0oq2Wi+r/VVRd1t7VVuxVkX7CugKagXbFW+A3Qq0ldCWBH1JoBIQSUAyA2Tm95lLQoKgdFdF4Xz/4ENmzjxzznmec55zeeZ8oQ7MjmDIy25uYpl68B0V5aR8XDhzmn3EQ/7wR33FQfgWEWPMEplH7yHDzRXc9BKXgBkDVgJAgeOAP/UaOJRlGczSmNFSA8K9ND/h4yeA3sNHY80Y1APuyN0xS8ChGcAV3J6OEtcvkPn1SPNgeX7YXoNDnLzl5irABPYsDCQYsGZwiYhxcOkN5hbrSihyJ103ABBj/1kcA2fFQMwJ+M1OgT8LermB95/iMBxwRjzwmUVLDQj32vwYfu9UqngMc++NYzw9PMP9xZrBb/QzjkEDhMSAzK9HmgfX8zBmR5mHQz8Sd8AAB1yI+moGz4HgNnw0iKti2KNuHvgjPwIQpod80JYzOcQjCKCF1w4LbAs04+A2fIzlKHFxXQzZOcI9Mz/rggbL4q6e3k+Od2gC1oGfIuNA1YHr0NES515gNmP8YZDI/HqsebAs6zF0lIuU5/HkN3CbjCCRP9WLHMiYzYB3hxgNvBvMKK3npjgHEc4Dx5sbgJEAPyMBiYezQ7/+VrePAjQR7vEA1MJswbIMLsHcIsa3MICZASRc05JIAfcfIJ4OhwEyv55pHiIXJ4Br6FOYOwZNnHlg3CwFHPo9hjk4AWNu7aCQO+nyCaXIPOEqYxUhjCtIgF9qAHB08cUdpSwAIHYThPtmf/wsmbMyhwEhLd4Y7sB/4giAS8HRRQbCx1DI/HqqeViJfh2DBzX3GYo5W8wDBycXd/ETeAtnAXInD8V0UqDJ8nl8iLMrMG68VliQOLvhDg6YyOWLdkER7osFAsvi/PCyd4DSU06aewE4A9vMz06kTtbvZ1Fd9UzzEIl+WcbJ1d1nwOPggIEbsC2ASUDi3MsSTtwdPonDu8F0Uoya4Ve0eoeMcpYCXc+yHq4YA7iLjMEl4sGeaHiIcN8skOHpex39+nkNDGnRQot7P8yBP2hc6gIADM+ug0YzPdY8hKNCWZb1johk9UA7uoIr3/kKk1ebYTFyJ10/neSD8XCWYZwUAxzJcU4GaGlpwDzA2d1HWIpAUxOE+72gAQyDOUicQsY7MBhm/K3FVSr1cZQ4S8W5C/oYvgebB696HAPMeehoR2/M6UZDk4Ozozc4urmBEEHeLcwD7x764hw7507MGI67Dx3j7AwOjcDQICUGiV+ZoV1QhPu3oGFzIHmvAUPdFOBEA1tHO/QLkXh4sdbtPWSBPdWh8B0UxmKspI/cY1CYkxlwIwW9JU7+QSI7Z7cwj+7gTqx7Jzg/s+w9OARz4GMnjODk48tvhPHzTWTXCPepv7AYIQbgFKhwCSAZM0hocHLGMA8f4aN5IRlCTxxtWDoojGHx3q5uA0PMLRiOgUOL2dHPH+N6Yax7mEc3mZ3YfjPkLFc2Y4C1QFMTSAMIxC2B8GCMkMEwhmGcvPxA1qeFguZbgHkFOPXuzZpb0KZdD7cNcY7CMLhE4tA3qIkCaIIWKUj7BXanL1vx7qQ2RpijuHqwfFxNszdAn/6iqhD9IsL9n6NgLIs5SNjefmCGZmdg+j6O4RIGLOsYyAJ7+DSFn8Zi7r4AYGaB8RuBu3sIHIDdwzy6iTsRPjbmT6rHmJZmphbAHdyk0HBkVyuVLxohItxP82Mtq15MQy3DgmugozlnH3W1VOLggDEMIJadnm0hApUGYBhrqjfrwSnAz/Hqv+u/OcLd6i7mgXcXt49ZZyeOnj4BL853dusjocCht6fwyQkyaIT7bX7CeheGYd7jZ/iMGobpmx09/fkVDqG3QOjRFsKbB44B9B48wn/+BNxQ7egqwR2c2G50ilu3Cp8VysJ/NAYNRXlMc4v7iDGtF9F3JwgPygKpCq3p1x9dh412lLlbPqIFZIE93jzEkW1LQ0N9wQVpf9JFMYARvoqH7tBBdRd3Ip6bJJJP8FteDtx8xWwGHMeRL0F4IOYn/jSbMQcHYfmLNZtxiQSZH7IQwTgw/nNXjGfwFDoonjirmxya0E1nJywrrFazvNqsZURNGuF+W6D1vB9+hxXHBAZGND9GsDcPjGX4WQluQ8KGZicPq87AZokB7Z0gPGjzA7S0hdDj7KS7zU6sGrIpIoahto3wgIefYGeAyPwQeoJ5oJOsEBAQEBDuAXBUBQgICAgIyJ0gICAgICB3cj9AGzUanemeSryh05TqjP/Zw41azS81NLKzRxFm/Z11R2s0GhNAs1Gn0VQ/hC1ByF6PaaciHlZ1PGjddUk9dLE7qczaM0uhIHlMmL/1VFl92xT6/FmKEfsrm2yv1Z5aHUQmXmlqT6I6ZeLEcd813bMc6vc/Hz5u4mPLjv8u7d5Ua2p5s8geMmzgkZL/PhsWge2D0aqv1T/iLUidOT08PCIiImDkyAgeJEl+/HPTPX9P5+uqavfwQXuLO9xapFWTJk0608RW7R8+blJmI/vAOhpd5xKK2etitRo0GYnTg4OFJh7x2s5MjeCg1QcnTYq8RLP3XK3XU6dGTkqtYdj72uKA1v3HQ8zO6C4qKuoM/Z+UwVoPVQdGREZl3GIerAGwXYfiTRMJgvjLF9m/arUV6ryPpwYQBPF5mdE+leHQkz6wNs/2ylceXrAtv32hpQeURPR39L3KI/XtBHfYq/qdT2lmBgWdbOIeP/GMFxzS/NfZsApsHxfHu04totlHGqUHCCJ899n8ojwrCirvQ5kujnebWkR1JmXVvihsl4rpWCnxyugcinmgtVSeGkgm3WI681Iue2cecPbaoC4vXqEMIhL+t0Ct1aq/z0h+miA8InbXMPc+VxcnyKYVmhiWrdofhcceaPhvX6GZpVScoMwdSak7PBVfe9l8v2pXM4OIPmP6T8TnPsPVg7mLFN517qT8FEEQc07rbP3E8Vgfz4hPb9gnrP36DQUR1+ohSlMJgthXwf1uupwxMyiI4PHyCY2tO6k8tgjmZzZauiqIT+H/p4oOLBfSQ/yW73nPZbicGhekEC6+eqTQvkN5SUw8P62RpX7YvUj46ReR+C8dzbLU2QVDB8fPfUqphF1Wl0N9E+MpJNteZjwe6wPxiSsmB/FPLT5dLSTRnnp9kpBm3JrMG227sbx3owKFu+NXpOlY2kYgzdYVfDw1zPZZayZHcX6rnQI+Ku5EQcT9cJs7z1s2lK98Dj/sngYxe+oozSp5wDMrlghaC7VUIFOet3Zyf4sxiOrQ5yY/pVRytRHz9sVq+u51VZX35vBAPn3ixjkK2FvcpkkXHUgg+UeWbF2jIOLOUMz1w7Ox+LRbDGdI8QqrIRXwD3I2I6SPWJGmZcQsPU1wWVIS0bvytHzZMxRkwo9C10+pF/Wf/R0nNgGLWbwuIZp/Ovz9PC1LFS/qzxVQQcy5RDH63NR4hZL/GXew2GDJ3mKSf9+SLWsUZFzXupPC132CiFW/2vbspac+OphTw7BsaZqCTLiiPk6QMcf0YoIbX6/2HvlpNcPoc9Nsi8aw7PXUBDyWqw2+dOHv5WltC1a1f7ZQ6lEH1df3TVWScesWiSl35lUxfIvLeiNKSDPu7Yy2/qwqb9Pk/oKaxiemahnqm1hv4ef7ahNTV/DJtGHCT/HZ0pTgYJI3m9QGxlSUkkiSws+k7wxtK7zpcuYspWgV84+XMnzZR5PRq1cvFp76855cYXxQlLI4mL+yeMvbSjKujTtpupz5olJBWuQI95jys2+NCCL5ivrisv76vtmChFEHSysPz8ZnpNabGZbSHk4YLpQ9IjG1lDMJ6tyi0LFL3k6czAn0i0j4pvzeGEqXuZNb/1oVRCSUtrl6JVlJROe0GZNSxTODguIsjqfkTU+IEXyDdlVAAGw7e5Oi9LmfEkR4lrHVnVTtm+QRkXzDItafTG7kPRNBhKdoqilKm7Wwr0dEci2rWRgY+OwJFcUJSSYI4rCtg6PUSX37wcrM6wbq5rFFBBH+lYp7Nuf1kP7EamtHv+Vs4VVj64CXKc96SjniiEZHUdTpF30IgviyUFevL+ZE8V4nf4G3b8TbKiPNlF/mirDL1odR52d5wdLMSpq6VZ43MyhowmmdjUD2/Cwv74gklZG+VZ4zMyhoySUjS2mT+vabe1pzk6JuL+CNR2l2QqRrjAYbNPIj3LggRey5arY8Q3SoVPHC/oFBRMLFamPdlcy4IMXkEzqW1W4I8Ie1mbUUVXU5jSCIDUUGtjyDJIi5pzUGgzZ9op8s4oCJ0ib167Cuahj622c8sJgdaoquupxGkkRbd1J6gCSJpLNa/pF+SiLuEsVU7YvyjEhuZDQL+/d/9nixiTckkiQOa5mbxxJIIjxFrb9VnjdLoYAv1ObyTJIk/8wnK85IJEmC60xLDxCkZSZBFQuTnqp9XBc5Nb2gltKfWRXmyWWP63ADyaQSvYHh5bx8XG2iqJLUxQQZfcrA8Nkjk3K0JkqbtUjMXtdp1HD0SS/sC3X7Y+zSFIKMyaGqk/r5Y+lCGsPJIZ7YzjxzeUawtWhpYtEEh8HVhkmojd3Vti6B0m7x95+bra6lqOupXMph9inzF/r6Ray+YqCY8surA+XYzgKbXFHnX/TBl2boKBOnJqViQna5uTxrNBmeXqo1Uez5F318IjZfqaNulefMUiqW5NcxLJW7Kgxfe6qyzlT79fJgMvxAaZWJ0mYv8udeZ1dg7erAQGxrzg0Tpc/dEUxGnKozM7w3giWpGkNdWebbwWQE5zn4i5tzyk0mTo6SnJ5vN8cQ5Jy5YTJxcoJ5OcLFNV/rTIYfkqNJMvrfddot/gFzs0prTVTV/sleI3c3mE0XXvJRkG98V0e1VBVs8Q/AVvyrhTF9O8U7mCD2Xi436ou5izsvt9yLKWOXuZPyz0MgJrXR/mLLxc3iUhUlolFcFhsq9oy8a3nuUrXosQ0GrVZbodaWFWfFBSnWlRht3Yks4oBldiK4E37paWnmTYrrqvS5nyqJ6LN0xQZ/f4elKT/o9I0dLnZpxGUra79fmsq7PZq7uC3vtoeK4xTP8tMpu6f0n0/k3AlfhMknNE0Gw03K8OMnUW06/fwF3v2J5VkqbWM7AlmW4oqs1aq12uIdIX154dTJP7ivK6HZ9gqYQz9K7sQe4cL6Xn1ukvC7795CcXihCFpySRyPX4h2h12FTOkBkiBSNNW8G9IeetIXdhVe3zfJPeLALXF5QvuTWs/V1RCPdSUUX1fesDSz1qauLhhK4hTKdzWUqPpnPDB7d1L+eRQnkLHMkslowZ24RxxoZHQbAgIclqYUtRoSdWKit1XCrXL1z9VU1b6oAJJzDOKKikKxodAguJNLbd3JVDn5qZiyNDkgOJl7r+Wfqn1Tg8jlxQZDvd5wqzwvXqHcUGhoN3tdp1HtxoCAdSpxuK7PTdu0Y8eePXt27Ei5qKcEd8LV3n6hmAxblRNMkl/qqHaLdl2oDaGnLt0jD05usOu1qa+f8FynEha7pnqNtDgbPqWpUTVLqZh8XE0LLe5Trp+17fTzF/oGkW+cKi63bXHxRPQloUO3aXE7Q/2xnZfNDKvfHYXvKjYz1MlJnCu6YarjrWgHSca0nVXY9FHxSuW64jrecwj+oHVdq3z3ZI+R+8VClaaRwTH27qR9OUIy/hlDWbG6ka+H9cVCPXACTY3FM5TEhoI6IUvXU6fzZeezvfV7wYUIZWm5F5bi0FVbdP6D/xy0+WwFzAi2uShx9wOARmg6OqJf4i1PAIg6UvTJULfBL77jtXtZxo35s39MzpVEHgv34ZMbsxY9tjS39/8NDn2u6nKRBP/j3V4qlbJE5tJhmZZ9IxjTC/zX5h9seHXO9LFrASDs3YP7XojoZfMIhUkAaOvz4j+Bg0PNxeU1tDcA+Lq3H9Nl81ZRlIW+04Nh8hc/M9Byvy7S3faNEe8XbVv+PwtixgFAv5h39m2bSdgI/CV18bPrL7AwKDSkpsjoAjJOOIXjfCadbi+gy6OzGc/CoJ1FRyY6sdZoKqkj99d11PxXGj/Y6TIq4y8h1poMkIm1OuDPI7EcHfUnrm7XPhOxVnzUbZRMijVLZdU6EwsuGIC7fAinKNpSV1Khrobb1JUzsM7AFtXQQEgBpI9NVQCN2dsPLxCAq9VAsh/LmlozH7A2L6XhtblxoiGlfPlCKHdDYxASuASSjwPoARp8pBZ1B8x8rP79GiPr2v6GbL2PrFeHteXuAEdjQ49aQmqwSCkHWY3OBKwLYHz2bIywCyAb7nkruqh6/UAZhoHEYKjUaExQc/DouZdGxoxyFbeI/SYulP5t4bmG1/6Yv52C5dH9nMxY26KN41oQXe/r5tzxl+MWtToD0AxI29Qb1+KWTBxkjeWJdO9lIyrivYJtia8smhLJstAveuOX22aSlhbHAvyatiRm/Xkzy7c4gzMrc7Zty1InVpm5bITFihgY7WJ3ppMxO2Hg0txe1waFzqi6XIjh40Rr8fWU8iJoqo1p9eZMixBMi21l/DVmLx649KIop8giB4OSOqHQIFMOlgnmzYr1YGe3wj99QmJlNcW1vGTwdRbEU/eOVrjL3InEL1gCn3/83frtom/gqvbcJ2/V+W54wsnJ7cK18fwlR5kb332P+7jXjWePn/BN33ftzdNPWmKBlub2/vCcJjoAAMriFeMbQdraFmkw+orVVKc62+gzjr+Ilb2Zzr4SYpeVwHFbT17bSht1eSnjXp75UUjJyoFOt2cYqzTbxHupiiR+r7tJ7xDbcYd+/DoumXOkaN1Qtw6cnvz5j79+HuhKzdkV418hw0LZFywCadXGNWdH7b94YEwAAP31kD7PtgaySgHY9gv4yLgTqafUCZxstCjE2JxK+syllz9bGP5hPps4Eri20hqv8mvW9+xYuTMUAPjsUV0aZ/Owdjfd4Osrdhy0TqWhlYPllroSjYF5JQSziaihARvjI4igfz12DWbZBcbQNK0bHCh2VVrVbxhm221hgeO2nrgqGFLkX2d9PLTwiUozkO7WkBtVk7s7Dbh1dAIVB391G+Mjw2iatRZarynE4Q7hOI1ic9HTkFikXuBt0xVoL9K6QYG9hN6Byx506WBC2nusU/81yZoX3h0AmFfU/I+iuHZalja+l62C/Ub+vffNTafPMZ/lun7wkReGVXNFW15UusAbby2b/nKn3niHFjc7vWBtiDvWQYt77qOTz/EtbuWEVweEhbZYWhxGq95Zey5i79n9YwIxjD71ZL9o+2g0ugm7ujK1+ZUwvF3R6uPL8ly2n/k1OhDHoOw54hlTB5m0N60SwbQwWzm5dnK4vo4zm0FOojz6qkrjHEy2Ww/WUYWhMKMFYrzuGy991wUKB8emTKnP/EvE6hNFRiPdUK05tXravCy3t3e96MV5ERGW5iob8/kqYuOSJcXEV/G2faWPCeqB1p2YElEkwVuMrd27q/wPClX25fqmZqPqwwXf1HGuWBqaEKPcsuZIWT0A/XPa3CAysUJ3giTJL4tqQCoLIAf37fDIGWnY+inKjStOVDQBGC+mJLfAhL5ud+gaVaW6DsJRpeTro43bXz1QwuWi+puXfLFl2Ta3dRsDAoYfLTKB1F8+eGyjyV6g1VDoXw68kmDyxmmuyM4M8+8ynam9AqofHXeCQ1GZpkLXCs21ehr0+TELj8YcKrqYv5PYMWvTd/XCYPOj6O1lNJj1+YfOuTwu94XgyEktv7380dlaALO+cHWg/NmTFYGRf/Wqfnfd+QoA47nXh02KTWtkMZu6ilZuWXPUpq5KnQJnyQyvv3tcB0AXZ6/TeLaxhr5hU4iStalF1QDGs5/+g2WllpzTrPZ4cDD5ZWG1xZB4m1k3Rblx5UmdsdmoWvPUH584qpFHvuxVs2n7dzo+Sj41X4I/JncDmS8OP1xW1XBit69nWWmHboAGWbXqWj3tG7lICv/YfF7DAhgL0oKDg/dV0Hz21h0WsrejNXtd5U7Grj8mh/99bMqaHI3eaDRWagqPJSbkS3BPbrBs7dBk47bMq1358orSEdvHBGAY+I5LcIb3Nl0o44uWLhQN7hbv6sww35dxM9HbYZYqlz1d/8FrKSojC3T1t7P74MuymNaWrnsnMHDEkcJbLN/iTJRti2NtW1zKq4sbPYG2tGuVpo52ClkUrdy27qiGE/1z+jxlcOKvbWNzxT7q5LRRhRg017c/BO0TOoUoWZ9aVM2C8dyOfzDs7SNaOzlcXxc8Nqr5p1c+OtvIUr9lJk6cMuVHEytlmO+v6oSwdRzoBim5bEjt/3vzCw3FgkFz8IvvK+MIT4zthoHCLGvIT15uXSl/Imbx4UL9HYLnNvj7twZr8euxabF+wrMvHck+9XooQRB7z3wlhoHV5a0KCBCW4Je/HW/ZvTdc2DzVujS/K1/P2sRrtRdnRX0T42mJ9DWceUsMx1IQc47zkV3c3XbCiLWbQ7lXryuptk0g7p2wLFunFqKihSAxMdzLWipLMJIQcaQy2go0/rB7tnhrZdq3qYkEQYQdKcnlM8YLv72Aj+7eCQG7Ll2IlsFSUenX9r8URCRcoUtmKoIcYmIsQXcpfHQUayrNtIb54UvThItlGastKotL13Aqs9YV025dlWYI0VkKIm75dEWftpFd1Jm3ooQHopYsUYhb8VP5yK7WIC7BkGpEm4kiRVXu+JHflKk8tpq0pEvKEbKpz3wpVLj25zUrBLHXD0/DYlIsGyHJniP5vZPyU0/zIWFnKKbyWJJVzoyDecLGy5lV4uus2etarTLlee9OH26tlkHxq0+qDcJWvJKMF7NXVzBLocBWnGqwZLYyI4m0lI0rGsNeT53WGv5busdrZLJ9KDCVuyqKJAnYWVx5uL2UdepPpslJscUtz9bbPazP3fO01eTik64YGJbVJg0LJAlibXFd0e45Ql5gRerp1ESSJEPTS6u/Xh1Mkh4Ru6vNhgtJ0yyZDd+Zd92+xvXpU/sKN19Mz/r6jTCSJL741yGu7OLGTLElJpjKWTVZkBO1eLGCjLPfO2lHzmdqk+lK5gxLX7Eiu5RhqLxVk4O5erjy276pYmRXnXrTs0GiEc5LLjUxLEt9O8Ub/+KKsF9yD/dOHoojIKtSZ49869auovSJbr//ix+aZqTSXr/rLk3TNuuJrZfaXOtIYqfSAU3DnVPd+ZW3v6dVIM3/K22bvvVSpzP5SIJWxQ+MHvuvn5cqoY4GD/tidqRb20t3q6v26rezJtCeVjt4RRvL7LxlmVr3Bv4bOQ9SZVye7tBOO3rmdz1ip9b73+LsWvjdbKKTBbmz7jqS0+Z6u/XQ6Q7uv8JDcqKwLils5OcGKQNjMkv2DnECBISO21zh04PjJ5wo2jBQhioDAeHhwcNzQD1dqdHdksrkct9eSC0Id4IxNy0bIqc85YPGHQgIyJ0gICAgIHQvoAPqERAQEBCQO0FAQEBAQO4EAQEBAQG5k3uN+0ancz/we6lpEInWQ4hOKuUR0B1d/egTRiEgd3IPca9pr+4HWqlp9g8bOymz084PkWg9jH1w9hPDBx4pYe9S1VyyQUc0D3W8ivqLyElpN1BIDQJyJyL+sECj0Yx7uCM/8+eFLivjPJ58UVWFen7nA5qdIiRwDz4gKkuYMP5S892zh9AppYTfQSllCc9MuNTEAkin1d1kXhmMPcwlkQawmBSF1yP0UHfyc1qiwPa5+NDeF8IXX2kCUKcpyMWqshMEGZNZIyarPbXaa+SOWoDqvLR4hZIkSSUZ/1VJPf8hfQIWu+Rvi2ME0tDtlyravOJq5panCUJ45GBbzmD63MKQP8xYsmCEgiRJLHbNZf5+c0GmlWn4ryfL+ISa1XL51HkzSJJ85x+zX7rq91PcoKe+KqtKnY3NSDfxBwumzQ4THnntaBEA/JaRgL1sYQJWp2AzDtpOYu78ivU/2+RTn79pcn+R9nhlegU0fRs7PF8iSRhIfnC1CQyFn0wbJtyNXHu8FkC/vzV7APQPKWINYzO2Xn6UpyzVeXsEPWKxa3JrmoTq2n1VWH2iv419kqsNWrM6UD5r5VIh5f+cvPTdnrmCYXx2qRqAvvCiz6CTZcLwXb9/Nv6V3aTEXikaTuyUEfk4njAo+IMy44UXfZ76quTcwj7k3/MsT+m2+AdsumQEWpf1xuRgixZu2Of8t4zFASPj/xKu9By55wbL2fAMpWiQh1RGzuCDE3/izxOsSl3cd+QKy/9z8GXZjWxNzuo5gmQsduv3RpYz2kWhQ2bMe5ogsF0qFqozEoYF89K2HTlaOdAZ9WUIXY+uOp0p6aymyaDeHBogHrHF85ScpWuS+vaDI+IZWSf/4A678gXepJdPaCiKKklNUBLRWUaRgnDqkcKblP7MW6FteaL4R5af0zSyhh8/iZITSbefxGVltdocGgBLM9sn46KKFwYGKoi4jEK1wdDKUiVwczWyVP4Cb57MyihQb71bRrdH28XTkOxV3fUVN6keT6LVlhWH06OF/8pXFnGAplRxQYotJQLZCXXkSa91JQaBUKs/sVplMJZlLOcPNMuspfRZC/tBTMotgdpkr0jTW/75JJ6y13pRu0rOKaWWVwpJhGcZGKY862lixBG1lqJEzhJj7molMUc4dIu9kkySXLL8BT5+EW8XG3gaNLkcdhXYUQTum00ShHRX1i9ansiLJGzZrk7rVXEK5ecaSiBWIQlin078Hzukrj02myTDj2sNTQZt+iQ/bFsew1DfxHqRBLElp+CqkariOQePaw1Med5quRyLsdCcICB0HbpgdqLN2V/ju3HJWMJRRr51eAvWel439ALveVtHBCRm1XLDyMIEk/e+mFB9zu4WmL58jG+zke4fMSvUXJzPzTYMNCTumR7iIfWNfDnBpQbsJvuBsT8X5i9TOlVqjC5kpCOc/8l+EYimsbIFKbNDAlx9By94a5hHvr4W5BtUKu0LRKOuusGdDDHrL1Zwo/o6HJ9yaG9sCCmT+T7p3dhfLhcOiWJAKqHLtp50W7n39UFubr6j5p/Pzp4X4CTcEjPTdinlLq/waE0vdZRh/TNzftJUY4EjU65dOz0hAPOT92V95fIAqRRGJ18tSpvlZtDWYr5jPOkPizQgtWYP8t7bWxa79tkAJ5qWhb64zrP6yE+P5hqYPueTGt+Nn04gZDJ53OGCiykTnTo4WrYOx+eeWDdI5qYcFIqBT9bbMZ5S3xGxcZ7V1N1eIt9QzCnFxCtlKMMpxbaqBbiGzRrFnN39gxEAinavLBu39hlpyUdfuw5bOy8YKINf8F9WPeG5p7DWLneciVb8z8TH5DJ9TnILO/2NMX4tnA3PDDWr8ip8Fg24+crP1SyU7Sl9Kj7a973vqlmo+LTU88sxcs8pn/14Jn0oGK8ZoU/kMI80VS1gvNEeWDE2VOEGl7eevbZm67NyGRY4csl7w2U1aGCM0PXoAr4Tu/XqwNBQ5qAtyY/fpEXSvy0417AgMv99GhKj/Z1aeKagmBALnY4E/6NIp9MxuZAhf9kTz59ydHw8JMRQWMhioe2c+G2hvXIPe969Or8W6nPbI+OyZWqyslRZZTgDW2EUr/iTJPD8SB2jfb4v21dY0WNJtNoYi8ApxCla4L+i7UKYnBkGLHXYmyfFsnITAYBncJhLDbRxQM5tHZIxa9HjS3NdRKXgrSRsdtxT0sErptWHHSzYHh78+ZHeiw+PxUDnwTB5NqRMN+1p0OxN9Ha2K1lYwlRIV918rOLgE/N+m1OzbXPRzSH095K4zf5OzQUHn3x+PcuyISEhRUW/3AyV9hKy7esunOzO+kugTA8sCRh4Dp7oiuIGEXqmO6FtLV+rKsQxu86Op9N59/Q5Zmeu60efeHEdtJ6GxB81C7xsx63/vuOoNmPzcceXvtW8G8xvYBCTDt0hcZ0quxkb530nMi5bF2LX79t6gkqNqt6ddGmPtktEZ18hvKeHkmjZmwrPfyX+0Kk0dDCJ2RyMWnHc6BbUCTnYb2aQitX76z8vsc+BrVKW5bp8eE4dHYABlM1QTrAqpY0PDvnr1qCp6eenDznhMOVYuAyaeBq09MJ1IZ05ibIdtitwjVe8cTDDQw/xH/kNqY4r2Pn1kZry+BXB0HTmrYSymUfNGyIxDPT7Jz/2yW0lqjTDID/BtdxUZTf4zEB9GUKXowsWu/qGTfWpXne4qIamdQcXJbBtl4RkkVvn1SbOS1SHf8h1l+AbmSCFv286XwY8UxBJkvsr7752w4AMbwKzPn/Rn1axIL1Jt5khscTGuScqmoDWXdx9snKcH99ndUjGZR0L88xLYKWmeWNIbdLc5LImoNTHx0yM/bSsqT3aLlvc5RUW9FwSLbs1S57/aq3IfxU2KTbtllQ2tLnp8KUSE9C/ZXxyysGxM1McrzEOcLqoAjg1rVR53ZbAxwQNnFKmjizEsRaB4IgtaUuA9oeJ86ij8+Zv/b835z+JAUjJZaONPCkTT4M22w9fltVRsK5v5CIp9ndbtqv9FTQEhr7aePqdf/7y/hg5Jh08yy1z/Wcl/y8u1MoNawLQFxx8akOpwddYa0egJx3w2sigtdsvGGkwqPYmfW/0QV0ZwkOArtiwaWWsWvrFzlHK6Tn8Vry4J8/T6cwMCoKVWVamrMpjm62USjMO5bMse/3wVAsjFsuWJvMb47bboHkLAwMFQpvks9nrw+RBRMJPtN1WfL+IaIGlSkHM4Qms2iHj2lNSsjAwcEsJLVL0WJiXqvZNgfg07o11xe9GCS8iRmzjM9wObZfAwaXqxCt6PInWbbiN/4r9eb9oPEHz3uarzsBS6pmKoO1l/EZ5aaqSiPuOsrBORSTfYtmmywcE+qC+EXM3zlHwO/DUNzFecEjFcErp00Ypn2s0m8PkXFWr9N/EeGFH1MI+9y+7p1n2zFmRBm2a3EqDlm1fzXb8V7wN38Z2xRZvmqQk5ggsUr/snqYk4s4ZuP/rc5OFxP2J5SfPHHyaIGB++olYH8xK50Wpk8LEVy9eHAM2L0JA6Cp0xYnCBs035zXExImEk7gS9UXJ8U58cXIP6XTokxP9o+f/m30hwGgEmcyOWOnOr+iYmqYN8U6HcjrPpYNItO5Uv7+7aHcixWpXKXclQLPJSycz87ts+C4sXr/XlhAQ7je6xJ0Uxg+Ny+0b+be5vlu3pZfFftC4PebBNgn6n+59pm/L6Qa7CwgICAg92J1wHkV34XxRFU1j8pDJEeSDH15VZqVd7DPuuRBfZAEICAgIj7I7QUBAQEDoXkAH1CMgICAgIHeCgICAgIDcCQICAgICcicICAgICAjInSAgICAgIHeCgICAgIDcCQICAgICcicICAgICAjInSAgICAgIHeCgICAgIDcCQICAgICcicICAgICAjInSAgICAgIHeCgICAgIDcCQICAgICcicICAgICAjInSAgICAgIHeCgICAgIDcCQICAgICcicICAgICAi/H/8/AAD//9V/Ft2JAwL7AAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A treasure-hunter agent\n",
    "This should be straightforward. The strength system works like this: monsters have strength between 1 and 8. The agent starts at strength 3, and gains 1 strength for every monster it defeats (and also it gets rewarded with treasure)\n",
    "\n",
    "Should be straightforward to implement; the Agent class stays mostly the same, just the rewards for certain interactions update. \n",
    "\n",
    "### Bellman Equation\n",
    "This is used to reward/punish the agent. The expected value can be positive (positive reinforcement!) or negative (negative reinforcement!) For our treasure hunter agent, we're rewarding the agent for getting to the exit and for defeating monsters and collecting treasure. We're punishing for actions taken to leave the environment, and also if the agent dies fighting a monster. So we need to balance the cost/benefit for encountering a monster so it knows to take the risk. \n",
    "\n",
    "![1_raYvY38yWm4E4J3iYqv6dA.png](attachment:1_raYvY38yWm4E4J3iYqv6dA.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TreasureHunter:\n",
    "    def __init__(self, environment):\n",
    "        self.actions = [0,1,2,3,4,5]\n",
    "        self.strength = 4\n",
    "        self.treasure = 0.0\n",
    "        self.environment = environment\n",
    "        self.env_size = environment.size\n",
    "        self.final_pos = (self.environment.size-1, self.environment.size-1, self.environment.size-1)\n",
    "        self.q_table = self.make_q_table(environment)\n",
    "        \n",
    "        # treasure hunter will encounter monsters\n",
    "        self.monster_encounters = 0\n",
    "    \n",
    "    def act(self, position):\n",
    "        x, y, z = position\n",
    "        \n",
    "        # might have to change this to pick randomly\n",
    "        action = self.choose_action(position)\n",
    "        \n",
    "        # update position\n",
    "        if action == 0: # up\n",
    "            new_pos = (x,y,z+1)\n",
    "        elif action == 1: # down\n",
    "            new_pos = (x,y,z-1)\n",
    "        elif action == 2: # left\n",
    "            new_pos = (x-1,y,z)\n",
    "        elif action == 3: # right\n",
    "            new_pos = (x+1,y,z)\n",
    "        elif action == 4: # forward\n",
    "            new_pos = (x,y+1,z)\n",
    "        elif action == 5: # backward\n",
    "            new_pos = (x,y-1,z)\n",
    "        \n",
    "        new_x, new_y, new_z = new_pos\n",
    "        \n",
    "        # check if agent went off the map\n",
    "        if new_x < 0 or new_y < 0 or new_z < 0 or new_x >= self.env_size or new_y >= self.env_size or new_z >= self.env_size:\n",
    "            reward = -5\n",
    "            new_pos = position\n",
    "            return action, new_pos, reward, False\n",
    "        \n",
    "        reward = 1\n",
    "        \n",
    "        # check for monsters and simulate interaction \n",
    "        monster = self.environment.map[new_pos][1]\n",
    "        if monster:\n",
    "            self.monster_encounters += 1\n",
    "            if self.strength > monster or (self.strength == monster and np.random.uniform() <= 0.5):\n",
    "                reward = 20 * self.environment.map[new_pos][0]\n",
    "                self.treasure += reward\n",
    "                self.strength += 1\n",
    "                \n",
    "                # clear out treasure and monster\n",
    "                self.environment.map[new_pos] = [0,0]\n",
    "                \n",
    "                return action, new_pos, reward, False\n",
    "            else:\n",
    "                reward = -1\n",
    "                return action, new_pos, reward, True\n",
    "        \n",
    "        return action, new_pos, reward, False\n",
    "            \n",
    "        \n",
    "        # Function for choosing the action for the agent\n",
    "    def choose_action(self, position):\n",
    "\n",
    "        # Selection of the action - 90 % according to the epsilon == 0.9\n",
    "        # Choosing the best action\n",
    "        if np.random.uniform() < 0.9 and not (np.argmin(self.q_table[position]) == np.argmax(self.q_table[position])):\n",
    "            state_action = self.q_table[position]\n",
    "            action = np.argmax(state_action)\n",
    "        else:\n",
    "            # Choosing random action - left 10 % for choosing randomly\n",
    "            action = np.random.choice(self.actions)\n",
    "        \n",
    "        return action\n",
    "\n",
    "    \n",
    "    \n",
    "    def learn(self, episodes=1000, learning_rate=0.9):\n",
    "        \n",
    "        self.learning_rate = learning_rate\n",
    "        \n",
    "        # reward decay\n",
    "        self.gamma = 0.1\n",
    "        \n",
    "        # reset q table\n",
    "        self.q_table = self.make_q_table(self.environment)\n",
    "        \n",
    "        for e in range(episodes):\n",
    "            position = (0,0,0)\n",
    "            self.strength = 6\n",
    "            self.treasure = 0\n",
    "            self.monster_encounters = 0\n",
    "            \n",
    "            # reset treasure and generate new monsters\n",
    "            self.environment.regenerate_hotspots()\n",
    "            \n",
    "            if e % 100 == 0:\n",
    "                print('episode {}'.format(e))\n",
    "            while position != self.final_pos:\n",
    "                action, new_pos, reward, dead = self.act(position)\n",
    "                if new_pos == self.final_pos:\n",
    "                    #### \n",
    "                    # Treasure hunter values exiting less\n",
    "                    ####\n",
    "                    self.q_table[position][action] += 2\n",
    "                else:\n",
    "                    ######\n",
    "                    # Bellman Equation! \n",
    "                    ######\n",
    "                    self.q_table[position][action] += learning_rate * ((reward + self.gamma * max(self.q_table[new_pos]) - self.q_table[position][action]))\n",
    "                # print(learning_rate, reward, self.gamma, max(self.q_table[new_pos]))\n",
    "                if dead:\n",
    "                    position = (0,0,0)\n",
    "                    break\n",
    "                \n",
    "                position = new_pos\n",
    "                \n",
    "    def run(self):\n",
    "        xy_run = np.zeros((self.env_size, self.env_size))\n",
    "        xz_run = np.zeros((self.env_size, self.env_size))\n",
    "        yz_run = np.zeros((self.env_size, self.env_size))\n",
    "        \n",
    "        xy_run[0,0] = 1\n",
    "        xz_run[0,0] = 1\n",
    "        yz_run[0,0] = 1\n",
    "        \n",
    "        position = (0,0,0)\n",
    "        self.strength = 6\n",
    "        self.treasure = 0\n",
    "        self.monster_encounters = 0\n",
    "\n",
    "        while position != self.final_pos:\n",
    "            action, new_pos, reward, dead = self.act(position)\n",
    "            # self.q_table[position][action] += reward\n",
    "            \n",
    "            x, y, z = new_pos\n",
    "            xy_run[x,y] = 1\n",
    "            xz_run[x,z] = 1\n",
    "            yz_run[y,z] = 1\n",
    "\n",
    "            if dead:\n",
    "                position = (0,0,0)\n",
    "                break\n",
    "                \n",
    "            position = new_pos\n",
    "        \n",
    "        return xy_run, xz_run, yz_run\n",
    "            \n",
    "    \n",
    "    # the q table is a dictionary containing the reward at each \n",
    "    # position in the environment\n",
    "    def make_q_table(self, environment):\n",
    "        \n",
    "        q_table = dict()\n",
    "        width, height, depth, size = environment.map.shape\n",
    "        \n",
    "        for i in range(width):\n",
    "            for j in range(height):\n",
    "                for k in range(depth):\n",
    "                    # index:action\n",
    "                    # 0:up, 1:down, 2:left, 3:right, 4:fwd, 5:bkwd\n",
    "                    q_table[(i, j, k)] = [0,0,0,0,0,0]\n",
    "        \n",
    "        return q_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 447,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "episode 0\n",
      "episode 100\n",
      "episode 200\n",
      "episode 300\n",
      "episode 400\n",
      "episode 500\n",
      "episode 600\n",
      "episode 700\n",
      "episode 800\n",
      "episode 900\n"
     ]
    }
   ],
   "source": [
    "env9 = Environment(size=9, hotspots=[(3,4,5,3), (1,4,1,3), (2,6,6,3)])\n",
    "\n",
    "th9 = TreasureHunter(env9)\n",
    "th9.learn()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A bad run\n",
    "Monster hunting is treacherous"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 452,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "monter encounters:  1\n",
      "0\n",
      "[[1. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 1. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 1. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]] \n",
      "\n",
      " [[1. 1. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 1. 1. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]] \n",
      "\n",
      " [[1. 1. 1. 1. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 1. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "xy, xz, yz = th9.run()\n",
    "\n",
    "print('monter encounters: ', th9.monster_encounters)\n",
    "print(th9.treasure)\n",
    "print(xy, '\\n\\n', xz, '\\n\\n', yz)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A good run\n",
    "\n",
    "Here's a good run, the treasure hunter was able to gain its strength and venture to the monsters' lairs and defeat them and gather some treasure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 453,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "monter encounters:  26\n",
      "5586.758516705193\n",
      "[[1. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [1. 0. 0. 0. 0. 1. 1. 1. 0.]\n",
      " [1. 1. 1. 0. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 1. 0. 0. 0. 1. 0. 0.]\n",
      " [1. 1. 1. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 1. 1.]] \n",
      "\n",
      " [[1. 1. 1. 0. 0. 1. 1. 0. 0.]\n",
      " [0. 0. 1. 0. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 1. 1. 1. 1. 1. 0. 0.]\n",
      " [0. 1. 1. 0. 1. 1. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 1. 1. 1.]] \n",
      "\n",
      " [[1. 1. 1. 1. 1. 1. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 1. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 1. 1.]]\n"
     ]
    }
   ],
   "source": [
    "xy, xz, yz = th9.run()\n",
    "\n",
    "print('monter encounters: ', th9.monster_encounters)\n",
    "print(th9.treasure)\n",
    "print(xy, '\\n\\n', xz, '\\n\\n', yz)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 459,
   "metadata": {},
   "outputs": [],
   "source": [
    "env20 = Environment(size=20, hotspots=[(3,4,5,3), (2,4,2,3), (2,6,6,3), (10,12,4,5), (5,16,9,3)])\n",
    "agent20 = TreasureHunter(env20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 460,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "episode 0\n",
      "episode 100\n",
      "episode 200\n",
      "episode 300\n",
      "episode 400\n",
      "episode 500\n",
      "episode 600\n",
      "episode 700\n",
      "episode 800\n",
      "episode 900\n",
      "episode 1000\n",
      "episode 1100\n",
      "episode 1200\n",
      "episode 1300\n",
      "episode 1400\n",
      "episode 1500\n",
      "episode 1600\n",
      "episode 1700\n",
      "episode 1800\n",
      "episode 1900\n",
      "episode 2000\n",
      "episode 2100\n",
      "episode 2200\n",
      "episode 2300\n",
      "episode 2400\n"
     ]
    }
   ],
   "source": [
    "agent20.learn(episodes=2500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 466,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "monter encounters:  227\n",
      "44335.55296750015\n",
      "[[1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 1. 1. 0. 1. 1. 1. 1. 0. 1. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 0. 0. 0. 1. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 1. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]] \n",
      "\n",
      " [[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 0. 1. 1. 1. 1. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 0. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 0. 0. 0. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 1. 0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
      " [0. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 0.]\n",
      " [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1.]] \n",
      "\n",
      " [[1. 1. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 0. 0. 1. 1. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 1. 1. 0. 0. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 1. 1. 0. 0. 0. 0. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 0. 0. 0. 0. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 1. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 1. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]]\n"
     ]
    }
   ],
   "source": [
    "xy, xz, yz = agent20.run()\n",
    "\n",
    "print('monter encounters: ', agent20.monster_encounters)\n",
    "print(agent20.treasure)\n",
    "print(xy, '\\n\\n', xz, '\\n\\n', yz)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nice! \n",
    "There were 233 monsters possible, so that run got close to killing them all and collecting *all* the treasure. This monster hunter is pretty good!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "learning",
   "language": "python",
   "name": "learning"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}