3_PythonBasics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JamesFergusson/Introduction-to-Research-Computing/blob/master/3_PythonBasics.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python\n",
    "\n",
    "Python is an modern object oriented coding language.  Its main strengths are its readability, its large array of optimised libraries and its interactivity making it an excellent place to begin learning to code.  It is fast becoming the default language for machine learning and data science and as such is the number one skill required by employers for data scientists. Here we will give a simple introduction to the language to get you started.  You should also look through a whirlwind tour of python here: https://github.com/jakevdp/WhirlwindTourOfPython\n",
    "\n",
    "\n",
    "## Installation\n",
    "\n",
    "Python should already exist on your department machine (depending on your department) and be easily accessable but possibly requiring you to load some modules.  See your local computer officer or web pages for more info.  You will also want to check the version available.  We will be working in *python 3* which has some significant changes from *python 2*\n",
    "\n",
    "To get it on your laptop or home computer there are several options but using the `conda` package and enviroment manager is probably the easiest. Just go to https://conda.io/miniconda.html and install Miniconda.  You could instead install Anaconda from https://www.anaconda.com/download/ which includes Miniconda and a bunch of other packages for all sorts of other things (and languages) it even comes with a graphical installer and graphical package manager for wusses.  The down sides are that it's quite large (3Gb) and you can install any of Anaconda's packages with Miniconda later on if you want them.\n",
    "\n",
    "Once installed you need to create an *enviroment*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conda create --name myfirstenviroment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will create an *enviroment* called `myfirstenviroment` (you should think of a better name).  An enviroment is just a container for all the bits for a project.  This keeps them seperate from your system so it's harder to break things.\n",
    "\n",
    "To access this enviroment you type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "activate myfirstenviroment (Win)\n",
    "source activate myfirstenviroment (Mac, Linux)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To leave the enviroment you type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deactivate (Win)\n",
    "source deactivate (Mac, Linux)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Documentation about *conda*  is available on the website: https://conda.io/docs/index.html.  For now you don't need to know much more than this.\n",
    "\n",
    "## Using python\n",
    "\n",
    "There are a few ways to use python.  The easiest is with jupyter notebooks (like this one!).  You will have to intall the module in your enviroment with conda,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "conda install jupyter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "then you can launch jupyter with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jupyter notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which brings up a window where you can create a notebook.  The terminal then becomes a window which lists all the background stuff happening in jupyter, you can stop this by running it in the background (sticking a `&` at the end).  You can then just type in code and run it (shift+return) in the window, a lot like in mathematica eg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "1+2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jupyter is good for fiddling about, creating plots and creating code with notes attached as it can handle basic latex for formulas, eg.\n",
    "\n",
    "$M_l(a,b,c) = \\int dx j_l(a x) j_l(b x) j_l(c x)$\n",
    "\n",
    "and it can even export the notebook as .tex, .html and .pdf for sharing or using in papers and talks.  But for larger projects you will probably want to run it using ipython which you start by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "$ ipython\n",
    "Python 3.7.0 (default, Jun 28 2018, 07:39:16) \n",
    "Type 'copyright', 'credits' or 'license' for more information\n",
    "IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.\n",
    "\n",
    "In [1]: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This brings up an interactive window where you can type and code.  You will probably want to run in one window with an editor with you code files (.py) in another.  \n",
    "\n",
    "Finally, once you have code you can just run it on the command line with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "python mywonderfullprogramme.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However as we will see you can do this in ipython too so there isn't much advantage to it unless you want it to happen in the background.\n",
    "\n",
    "In this course I would recommend you work in ipython as this is where you will most likely end up working the most as it sets you up to run your code in the batch (fancy word for background) but feel free to use Jupyter instead if you want (you can export your notebook as a .py file but this doesn't turn it into runable code unfortunatly).\n",
    "\n",
    "With that, let's begin to code!\n",
    "\n",
    "\n",
    "## Data types\n",
    "\n",
    "The very first this we need to learn before we can code is what objects does the code support?\n",
    "\n",
    "The simple types in python are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = 1          # Integer\n",
    "b = 1.0        # Floating point number, usually refered to as a 'float'\n",
    "c = 1e0        # Again a float but in enginering notation\n",
    "d = 1.0 + 2.0j # Complex numbers, which follow enginering and use a 'j' (boo!)\n",
    "e = True       # Boolean which just means True or False\n",
    "f = 'Hello'    # Strings which just means text\n",
    "g = None       # None type, a special type which just means no type or 'null'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python changes types dynamicaly in the backgound so you can do the following without any problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "x = 'abc'\n",
    "x = True\n",
    "x = 5e12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This also protects you from some gotchas from other explicit languages like overrunning the allowed range for say a 32bit integer (which runs from -2,147,483,647 to 2,147,483,648 so two billion plus one billion ends up negative). Python just allocates more and more memory to cope with this,\n",
    "\n",
    "There are all the usual standard operations for use with numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5\n",
    "y = 2\n",
    "print(x + y)  # Addition\n",
    "print(x - y)  # Subtraction\n",
    "print(x * y)  # Multiplication\n",
    "print(x / y)  # Division\n",
    "print(x // y) # Floor function\n",
    "print(x % y)  # Modulus\n",
    "print(x ** y) # Exponentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: exponentiation is NOT `^` which is bitwise XOR.  Bitwise operators {`&`,`|`,`^`,`<<`,`>>`,`~`} operate on integer's binary parts directly and I have never seen, nor can imagine, a use for them in data science so you can forget them.  They are mostly used for very low level data processing.\n",
    "\n",
    "All the operations have a short version for self assignment as it's so common. So instead of `x = x + y`  we can use `x += y`.  All of the operators support this.\n",
    "\n",
    "There are a couple of things to watch out for.  Firstly division upcasts integers to floats (only in python3).  This can be overcome by forcing the cast to `int` or using floor division. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(4/2)\n",
    "print(4//2)\n",
    "print(int(4/2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, as with all computing, operations on floats are NOT exact"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(0.1 + 0.2 == 0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So tests for floats should always use a tolerance, eg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "abs(0.1 + 0.2 - 0.3) < 1e-10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\+ and \\* even work as you would expect with strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = 'hello '\n",
    "b = 'world'\n",
    "print(a+b)\n",
    "print(3*a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But not any of the other operations.  For operations on strings we instead have lots of functions.  How do we find out information on them?\n",
    "\n",
    "\n",
    "## Getting help\n",
    "\n",
    "Rather than list all possible function for every data type I will instead show you how to find the information for yourself.  Firstly, as with all coding, googling is always a good idea and the stackoverflow website is particularly heplful.  \n",
    "\n",
    "However, ipython and jupyter both have someting called tab completion (which we already met in our section on the terminal) which is very handy here (and just for typing code in general).   Type the following and then push `tab` (cells share variables so this one knows that `a = hello` already)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This brings up a list of all possible functions that can be applied to the string object `a`.  We should quickly pause and make note of the structure of python.  Python is an 'object oriented' code meaning that is it organised around objects rather than actions.  This presents itself mainly in that when we want to perform an action on an object `a` rather than typeing `action(a)` we type `a.action()` with the `.` binding the action to the object.  There will still be lots of times when we do end up writing `action(a)` for functions that can take multiple inputs but to find the oprtations that explicitly work on `a` will all be of the format `a.thing()`   Now back to where we were.\n",
    "\n",
    "Say we want to choose the first `capitilize` try pushing tab after each of the following"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a.c\n",
    "a.ca\n",
    "a.cap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last will auto complete to `a.capitalize`.  We can likely guess what it will do but python has another excellent feature, `?`, which provides documentation on what commands do.  Try"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a.capitalize?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can even go further and use `??` which brings up the documentation and the python code behind the command.  This is mostly useful for your functions as most inbuild python functions are not written in python so it only brings up the documentation just like `?`.\n",
    "\n",
    "Now to use it we must type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a.capitalize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These two features make python particularly easy to work with as you can easily find out all possible operations and how to use them from the command line as you code!\n",
    "\n",
    "Finally, one function which is useful for strings is `.format` which allows you to replace parts of strings denoted by `{}` with formated input.  It has lots of options but here are some examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pi is: 3.14\n",
      "a, b, c\n",
      "The complex number (1+2j) is formed from the real part 1.0 and the imaginary part 2.0.\n",
      "Coordinates: 37.24N, -115.81W\n",
      "list element 0: 1;  list element 3: cat\n",
      "1,234,567,890\n"
     ]
    }
   ],
   "source": [
    "pi = 3.14159265\n",
    "list = [1,2,3,'cat',True,3.1415]\n",
    "\n",
    "str1 = 'Pi is: {:.2f}'.format(pi)\n",
    "str2 = '{}, {}, {}'.format('a', 'b', 'c')\n",
    "str3 = 'The complex number {0} is formed from the real part {0.real} and the imaginary part {0.imag}.'.format(d)\n",
    "str4 = 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W')\n",
    "str5 = 'list element 0: {0[0]};  list element 3: {0[3]}'.format(list)\n",
    "str6 = '{:,}'.format(1234567890)\n",
    "print(str1)\n",
    "print(str2)\n",
    "print(str3)\n",
    "print(str4)\n",
    "print(str5)\n",
    "print(str6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As of python 3.6 you can now also use the following syntax for formatting which is simpler:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pi is: 3.14\n",
      "a, b, c\n",
      "The complex number (1+2j) is formed from the real part 1.0 and the imaginary part 2.0.\n",
      "Coordinates: 37.24N, -115.81W\n",
      "list element 0: 1;  list element 3: cat\n",
      "1,234,567,890\n"
     ]
    }
   ],
   "source": [
    "pi = 3.14159265\n",
    "list = [1,2,3,'cat',True,3.1415]\n",
    "latitude='37.24N'\n",
    "longitude='-115.81W'\n",
    "num = 1234567890\n",
    "\n",
    "str1 = f\"Pi is: {pi:.2f}\"\n",
    "str2 = f\"{'a'}, {'b'}, {'c'}\"\n",
    "str3 = f\"The complex number {d} is formed from the real part {d.real} and the imaginary part {d.imag}.\"\n",
    "str4 = f\"Coordinates: {latitude}, {longitude}\"\n",
    "str5 = f'list element 0: {list[0]};  list element 3: {list[3]}'\n",
    "str6 = f'{num:,}'\n",
    "print(str1)\n",
    "print(str2)\n",
    "print(str3)\n",
    "print(str4)\n",
    "print(str5)\n",
    "print(str6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that you need to have the quotes around the string and around elements to be different, i.e. single vs double.  It does not matter which way around you do it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "1. Calculate sin(0.1) using its taylor expansion to order 5\n",
    "2. Print the result as a descriptive string stating the order expanded to and value to 5 decimal places"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data structures\n",
    "\n",
    "There are also 4 inbuilt data structures in python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list = [1,2,3,'cat',True,3.1415]     \n",
    "tuple = (1,2,3,'cat',True,3.1415)    \n",
    "dict = {'one':1,'two':2,'three':3}   \n",
    "set = {1,2,3,4,5}                    \n",
    "print(list)\n",
    "print(tuple)\n",
    "print(dict)\n",
    "print(set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists are just that, lists of objects of any type, including lists.  Tuples are the same but they are immutable meaning you can't change them once created.  Why would you want something immutable? They are normaly used for returning results from functions like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 0.125\n",
    "x.as_integer_ratio()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists however you can change as much as you want.  List indexing is `0` based meaning that `0` gives the first entry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and `3` will give the fourth element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also start from the other end using negative numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list[-1])\n",
    "print(list[-3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can pick out sections using `:`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list[2:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "leaving out the first or last part assumes you meant to the beginning or end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list[:3])\n",
    "print(list[3:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can even walk through with a given step size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list[::3])\n",
    "print(list[1:5:2])\n",
    "print(list[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can of course both change the entries and add more"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list[3] = 'dog'\n",
    "list.append(100)\n",
    "print(list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One other useful list function is `len(list)` which gives the length of the list. Try `list.<tab>` to see what else you can do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings are just lists of letters so all the same indexing works for them too.  However strings are immutable like tuples so the following doesn't work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a[2] = 's'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One final thing is that while you can have lists of lists these are NOT arrays.  Even if the following looks OK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat = [[1,2,3],[4,5,6],[7,8,9]]\n",
    "mat[1][2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that more complicated commands go a bit wrong (`:` just means 'all')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(mat[1][:])\n",
    "print(mat[:][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Any idea what is happening?\n",
    "\n",
    "We will see later a much better way of handling arrays when we use the package `numpy`\n",
    "\n",
    "Dictionaries are just like lists but their contents are indexed by their key (bit before the colon) rather than position"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict['two']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can add new entires like so"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict['four'] = 4\n",
    "print(dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionaries keep no sense of order (which makes them efficent to search)\n",
    "\n",
    "Sets are like lists but the items are unordered and unique.  You have the usual operators you would expect "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set2 = {4,5,6,7,8,9}\n",
    "print(set & set2) # Intersection\n",
    "print(set | set2) # Union\n",
    "print(set - set2) # Difference\n",
    "print(set ^ set2) # Symmetric difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally there is one sneaky trap that comes with data structures which is visable in the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = [1,'cat',3.1415927,dict]\n",
    "list2 = list1\n",
    "print(list2)\n",
    "list1[1] = 'BOO!'\n",
    "print(list2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is because the data objects we define are just addresses (or pointers) which label a location in memory. The detail of what is happening for list is actually rather complicated.  In creating `list1` the computer does the following (this is actually a simplification but will do for here):\n",
    "\n",
    "![title](Plots/ListMemory.jpg)\n",
    "\n",
    "Then when we assign `list2 = list1` all that happens is that `list2` is assigned to `addr1` too.  Then when we change element 1 via `list1[1] = 'BOO!'` we change `addr3` to `addr6` which will store the new string `'BOO!'` (`addr3` will now be released and cleaned up by python's garbage collector).  When we print list2 it goes to `addr1` and outputs what it finds which includes the new second element.  \n",
    "\n",
    "This approach makes lists very flexable so they can store anything at all, even functions.  The idea of just pointing list2 to list1 is to save memory.  If you want a copy of list1 assigned to list2 you have to do this: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list2 = list1.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But beware this is only a shallow copy so list of list don't copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = [[1,2],3]\n",
    "list2 = list1.copy()\n",
    "print(list2)\n",
    "list1[0][0] = 'BOO!'\n",
    "print(list2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you need to do a deep copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import copy\n",
    "list2 = copy.deepcopy(list1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Control Flow\n",
    "\n",
    "Now we come to the two basic building blocks of all code: commands that perform a test and commands that perform a loop.\n",
    "\n",
    "First the control structure for performing a test is `if`. The format is:\n",
    "\n",
    "`if` 'condition' `:`\n",
    "\n",
    "`*spaces*` 'commands'\n",
    "\n",
    "`elif` 'condition' `:`\n",
    "\n",
    "`*spaces*` 'commands'\n",
    "\n",
    "`else` 'condition' `:`\n",
    "\n",
    "`*spaces*` 'commands'\n",
    "\n",
    "The condition is anything that evaluates to a boolian value (`True` or `False`) and all the comparison operators are:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a == b # Equal\n",
    "a != b # Not Equal\n",
    "a < b  # less than\n",
    "a > b  # greater than\n",
    "a <= b # less or equal\n",
    "a >= b # greater or equal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use numbers, where 0 is `False` and all others are `True`. Also string and lists where empty ones are `False` and populated ones are `True`.  `None` types are always `False`.  \n",
    "\n",
    "Here is an example which makes it pretty clear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 20\n",
    "\n",
    "if x==0:\n",
    "    print('x is zero')\n",
    "elif x>0:\n",
    "    print('x is +ve')\n",
    "elif x<0:\n",
    "    print('x is -ve')\n",
    "else:\n",
    "    print(\"Don't Know\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the last `print` we have used double quotes `\"` so we can use single quotes `'` (an apostrophe) inside without an error.\n",
    "\n",
    "This is the first time we see the python syntax in action.  Here are the basics:\n",
    "\n",
    "- \\# denote comments. Everything on the line after them is ignored\n",
    "- indentation indicates a code block and always follows a `:` on the preceding line. Think of it like 'brackets'. The amount of indentation is up to you as long as you are consistent but 4 spaces is the convention.\n",
    "- Spacing inside lines is irrelevent.\n",
    "- Lines are terminated by a carrage return but can be continued by adding a `\\` at the end or by having parentheses, `()`\n",
    "- Multiple statements can be on the same line if you use `;` between them.\n",
    "\n",
    "Here are examples of the last two"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=1; y=2\n",
    "z = x + \\\n",
    "y\n",
    "print(z)\n",
    "z = (x + x +\n",
    "     y + y)\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second control structure, loops, is much more varied.  You have two basic commands `for` and `while`.  `for` has the format:\n",
    "\n",
    "`for` 'iterator object' `:`\n",
    "\n",
    "`*spaces*` 'do a bunch of stuff'\n",
    "\n",
    "While has the format:\n",
    "\n",
    "`while` 'condition' `:`\n",
    "\n",
    "`*spaces*` 'do a bunch of stuff'\n",
    "\n",
    "Let's see some examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = [1,2,3,4,5]\n",
    "\n",
    "print('For loop')\n",
    "for item in list1:\n",
    "    print(item)\n",
    "\n",
    "print('\\nWhile loop')\n",
    "x=1\n",
    "while x<6:\n",
    "    print(x)\n",
    "    x+=1\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`for` loops are safer than `while` loops as they will generally end, if we had forgot the `x+=1` in the while loop it would go on forever (`\\n` is a special character meaning newline, the other main one is `\\t` meaning tab). \n",
    "\n",
    "The 'iterator object' part is actually pretty rich with alot of possibilities. An iterator is anything which can generate a sequence or list of objects.  Its value is that the list never needs to be created so saves memory.  Two of the most common are `range` and `enumerate`.  `range` just gives you numbers in order and `enumerate` returns the item and its index from a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = ['apple','pear','banana','mango']\n",
    "\n",
    "print('enumerate')\n",
    "for index, item in enumerate(list1):\n",
    "    print(index,':',item)\n",
    "\n",
    "print('\\nrange1')\n",
    "for n in range(5):\n",
    "    print(n) \n",
    "\n",
    "print('\\nrange2')  b\n",
    "for n in range(5,10):\n",
    "    print(n)  \n",
    "\n",
    "print('\\nrange3')  \n",
    "for n in range(0,12,3):\n",
    "    print(n) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loops can be fine tuned with two further commands `continue` and `break`.  `continue` sends you to the next interation, ignoring the rest of the loop.  `break` ends the loop.  Here is an example with both:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n=0\n",
    "while True:\n",
    "    n+=1\n",
    "    if n>30:\n",
    "        break    \n",
    "    if n%3==0:\n",
    "        continue\n",
    "    print(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes you might want to iterate over multiple lists at once.  Here you can use the function `zip` to join the list together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xvec = [1,2,3,5,7,8,9]\n",
    "yvec = [7,4,6,7,6,4,6]\n",
    "\n",
    "for x,y in zip(xvec,yvec):\n",
    "    print(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "zip is useful for joining data, but it is also its own inverse:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xvec = [1,2,3,5,7,8,9]\n",
    "yvec = [7,4,6,7,6,4,6]\n",
    "\n",
    "z1 = zip(xvec,yvec)\n",
    "print(*z1)\n",
    "z1 = zip(xvec,yvec)\n",
    "z2 = zip(*z1)\n",
    "print(*z2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This contains some subtilties.  We have to use `*z1` in print as `z1` is a generator for a list rather than an actual list (to save memory) and the `*` tells it to evaluate it.  Also generator objects can only be used once (we'll see why later), so after printing `*z1` we have to define it again before creating `z2`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions\n",
    "\n",
    "Now that we have the ability to write basic code we need a way create functions for frequently used operations.   There are two types of functions explicit ones and anonomous ones.  First here is how to define functions explicitly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add2(x,y):\n",
    "    return x+2*y\n",
    "\n",
    "print(add2(1,2))\n",
    "print(add2('abc','def'))\n",
    "print(add2([1,2],[3,4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the functions you don't define types for the input so here numbers, strings and lists are all fine as inputs.\n",
    "\n",
    "You can also define default arguments in functions like this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fibonacci(N, a=0, b=1):\n",
    "    list = []\n",
    "    while len(list) < N:\n",
    "        a,b = b,a+b\n",
    "        list.append(a)\n",
    "    return list\n",
    "\n",
    "print(fibonacci(5))\n",
    "print(fibonacci(5,1))\n",
    "print(fibonacci(5,2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: the part `a,b = b,a+b` means compute the pair `b,a+b` then assign them to the pair `a,b`.  Here is one gotchya!  to watch out for:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def addlist(x,A=[]):\n",
    "    A.append(x)\n",
    "    return A\n",
    "\n",
    "a = addlist(1)\n",
    "print(a)\n",
    "b = addlist(3,[1,2])\n",
    "print(b)\n",
    "c = addlist(2)\n",
    "print(c)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here `A=[]` first allocates an empty list to `A` <b>when the function is created</b> not when it is run. Now whenever you edit `A` it's storded at the same address so is kept for subsequent calls to the function.  So as we have pointed `a` and `c` to the same list editing one edits both.\n",
    "\n",
    "The following would fix this behaviour:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def addlist(x,A=[]):\n",
    "    B = A.copy()\n",
    "    B.append(x)\n",
    "    return B\n",
    "\n",
    "a = addlist(1)\n",
    "print(a)\n",
    "b = addlist(3,[1,2])\n",
    "print(b)\n",
    "c = addlist(2)\n",
    "print(c)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You could also solve this using the following which avoids the extra copy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def addlist(x,A=None):\n",
    "    if A is None: A=[]\n",
    "    A.append(x)\n",
    "    return A\n",
    "\n",
    "a = addlist(1)\n",
    "print(a)\n",
    "b = addlist(3,[1,2])\n",
    "print(b)\n",
    "c = addlist(2)\n",
    "print(c)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes you may want a function that can handle an unknown number of arguments, like computing statistics of numbers.  For this you can use the special case `*args` which takes multiple arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def h_mean(*args):\n",
    "    x = 0e0\n",
    "    for item in args:\n",
    "        x+= 1e0 / item\n",
    "    x = len(args)/x\n",
    "    \n",
    "    return x\n",
    "\n",
    "print(h_mean(5))\n",
    "print(h_mean(5,6))\n",
    "print(h_mean(5,6,7,8,9))   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can do the same with dictionarys of unknown length. These are called as keyword arguments and are implemented with `**kargs`. For example, checking a list of planets to see which closest to a particular size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def planet_check(radius,**planets):\n",
    "    x = 1e100\n",
    "    for key, val in planets.items():\n",
    "        if abs(radius-val)<x:\n",
    "            x=abs(radius-val)\n",
    "            best = key\n",
    "    print('The planet which is closest in size is '+best)\n",
    "    return None\n",
    "\n",
    "planets = {'Mercury':2440,'Venus':6052,'Earth':6378,'Mars':3396,'Jupiter':71492,'Saturn':60268,'Uranus':25559,'Neptune':24764}\n",
    "\n",
    "planet_check(5000,**planets)\n",
    "planet_check(10000,**planets)\n",
    "planet_check(20000,**planets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes we might want to use a small function once in a throw-away manner.  For these instances there are anonomous or 'lambda' functions.  These are mostly used as input to other functions.  The format is:\n",
    "\n",
    "`lambda` 'input stuff' `:` 'output' \n",
    "\n",
    "Here are some examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test1 = lambda x : x**2 + x + 1\n",
    "test2 = lambda x,y,z : x*y+z\n",
    "test3 = lambda x: x%3==0\n",
    "\n",
    "print(test1(5))\n",
    "print(test2(1,2,3))\n",
    "print(test3(7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One good example is non-trivial sorting, say for absolute size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list = [-1,-2,-3,-4,-5,2,3,4,5,6,7,8]\n",
    "list.sort()\n",
    "print(list)\n",
    "list = sorted(list,key=lambda x: abs(x))\n",
    "print(list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercises\n",
    "\n",
    "3. Construct a function which returns a list of prime numbers less than a given integer, N\n",
    "4. Construct a function which returns a list of the first N terms in the Recaman's sequence (http://mathworld.wolfram.com/RecamansSequence.html)\n",
    "5. Compute a list of the numbers which appear in both lists when they are both N items long "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comprehensions and Generators\n",
    "\n",
    "Python contains some neat ways for creating lists the first is a 'list comprehension'.  This is a way to create a list from a rule rather than listing the objects directly.\n",
    "\n",
    "Its format is the following:\n",
    "\n",
    "`[`'object to go in list' `for` 'input' `in` 'list/generator' `if` 'condition'  `]`\n",
    "\n",
    "Here are some examples to show you what is looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print( [x**2 for x in range(10)]                            ) # Squares\n",
    "print( [x for x in range(10) if x%2==0]                     ) # Evens\n",
    "print( [x if x%2 else -x for x in range(10)]                ) # Evens -ve Odds +ve\n",
    "print( [x**3 for x in [1,2,4,8]]                            ) # Cubes from a list\n",
    "print( [(x,y) for x in range(5) for y in range(5) if x+y<4] ) # Create pairs with sum<4\n",
    "print( [x**2 for x in [x**2 for x in range(10)]]            ) # Pointlessly complicated 4th power"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generators are very similar to list comprehensions but with two key differences\n",
    "- List comprehensions create the list explicitly, generators are just recipies for creating lists.  This allows you to loop over lists without the memory overhead of actually creating them\n",
    "- Generators are single use, once you use them they are gone.  This does mean you can pause them halfway then restart where you left off\n",
    "A generator looks just like a list comprehension but with `()` rather than `[]`.  If we want the explicit list we can call `list()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = [x**2 for x in range(10)]\n",
    "gen1  = (x**2 for x in range(10))\n",
    "print(type(list1))\n",
    "print(type(gen1))\n",
    "print(list1)\n",
    "print(gen1)\n",
    "print(list(gen1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is an example of how you can pause them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gen1  = (x**2 for x in range(10))\n",
    "\n",
    "for n in gen1:\n",
    "    print(n)\n",
    "    if n>20:\n",
    "        break\n",
    "\n",
    "print('*have a quick lie down*')\n",
    "\n",
    "for n in gen1:\n",
    "    print(n)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But you can't use them twice!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gen1  = (x**2 for x in range(10))\n",
    "\n",
    "for n in gen1:\n",
    "    print(n)\n",
    "\n",
    "print('And again?')\n",
    "\n",
    "for n in gen1:\n",
    "    print(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some generators you may want to use may be too complex for a single line.  In this case we can create a generator function.  This is just a normal function but with the command `yield` in it. `yield` can be in the function multiple times and whenever we encounter it we return the value next to `yield`.  Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gen1():\n",
    "    a = 'who'\n",
    "    b = 'let'\n",
    "    c = 'the'\n",
    "    d = 'dogs'\n",
    "    e = 'out,'\n",
    "    for i in range(3):\n",
    "        yield a\n",
    "        yield b\n",
    "        yield c\n",
    "        yield d\n",
    "        yield e\n",
    "        for j in range(3):\n",
    "            yield a\n",
    "        yield '?\\n'\n",
    "\n",
    "G1 = gen1()\n",
    "print(*G1)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "6. Create a list of all pairs of factors (as tuples) of 362880 using list comprehension.\n",
    "7. Write a generator function for a random walk, step size 1, which is equally likely to go up or down.  End the generator when you have total displacement of 10 steps (you will need a random number generator like `random.randint(a,b)` which gives a random integer between a and b inclusive, you will need to the line `import random` in order to use it)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Magic commands\n",
    "\n",
    "Python also has a number of 'magic' commands which are preceded by a % or %%. These are for use in the interpreter (ipython or jupyter) to do useful things outside of coding.  There are load of them (try `%magic` which lists them all) but here are some of my favorites:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%whos # list of all assigned variables\n",
    "%history -n 1-4 # list commands from prompts 1-4\n",
    "%run filename.py # runs the python script filename.py\n",
    "%timeit # times one line of code\n",
    "%%timeit # times multiple lines of code\n",
    "%debug # opens a debugger where an exception was raised\n",
    "%rerun # rerun previously entered commands (can specify range)\n",
    "%reset # delete all varible and definitions\n",
    "%save # save some lines to a specified file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Play around with them, we will come back to these when we get to the section on performance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}