Traffic_Sign_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Self-Driving Car Engineer Nanodegree\n",
    "\n",
    "## Deep Learning\n",
    "\n",
    "## Project: Build a Traffic Sign Recognition Classifier\n",
    "\n",
    "In this project, deep neural networks and convolutional neural networks are used to classify traffic signs. Specifically,we will train a model to classify traffic signs from the German Traffic Sign Dataset and later use this model to predict the sign in the images from web."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Step 0: Load The Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load pickled data\n",
    "import pickle\n",
    "import cv2\n",
    "\n",
    "# Filling based on where the training and testing data is saved\n",
    "\n",
    "training_file = 'train.p'\n",
    "validation_file= 'valid.p'\n",
    "testing_file = 'test.p'\n",
    "\n",
    "with open(training_file, mode='rb') as f:\n",
    "    train = pickle.load(f)\n",
    "with open(validation_file, mode='rb') as f:\n",
    "    valid = pickle.load(f)\n",
    "with open(testing_file, mode='rb') as f:\n",
    "    test = pickle.load(f)\n",
    "    \n",
    "X_train, y_train = train['features'], train['labels']\n",
    "X_valid, y_valid = valid['features'], valid['labels']\n",
    "X_test, y_test = test['features'], test['labels']\n",
    "\n",
    "#print(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 1: Dataset Summary & Exploration\n",
    "\n",
    "The pickled data is a dictionary with 4 key/value pairs:\n",
    "\n",
    "- `'features'` is a 4D array containing raw pixel data of the traffic sign images, (num examples, width, height, channels).\n",
    "- `'labels'` is a 1D array containing the label/class id of the traffic sign. The file `signnames.csv` contains id -> name mappings for each id.\n",
    "- `'sizes'` is a list containing tuples, (width, height) representing the original width and height the image.\n",
    "- `'coords'` is a list containing tuples, (x1, y1, x2, y2) representing coordinates of a bounding box around the sign in the image. **THESE COORDINATES ASSUME THE ORIGINAL IMAGE. THE PICKLED DATA CONTAINS RESIZED VERSIONS (32 by 32) OF THESE IMAGES**\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Summary of the Data Set Using Python and Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples = 34799\n",
      "Number of testing examples = 12630\n",
      "Image data shape = (32, 32, 3)\n",
      "Number of classes = 43\n",
      "Number of validation examples= 4410\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Number of training examples\n",
    "n_train = len(X_train)\n",
    "\n",
    "# Number of validation examples\n",
    "n_validation = len(X_valid)\n",
    "\n",
    "# Number of testing examples.\n",
    "n_test = len(X_test)\n",
    "\n",
    "# The shape of an traffic sign image\n",
    "image_shape = X_train[0].shape\n",
    "\n",
    "# Unique classes/labels in the dataset.\n",
    "n_classes = len(np.unique(y_train))\n",
    "\n",
    "print(\"Number of training examples =\", n_train)\n",
    "print(\"Number of testing examples =\", n_test)\n",
    "print(\"Image data shape =\", image_shape)\n",
    "print(\"Number of classes =\", n_classes)\n",
    "print(\"Number of validation examples=\", n_validation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Include an exploratory visualization of the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the German Traffic Signs Dataset using the pickled file(s) by plotting a bar graph of classes vs number of images in each class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABZCAYAAABR/liSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAEeJJREFUeJztnFmMHNd1hr9bSy/Ts5McrjJJiVqolbIpylosM5JsLU4gB3CMOLDhAAGUFwMJkIcYecqjH5IAeYkBBXEQJAYCB05sB2ZsWbIky4rERRRJcRvu+3DIWbqn99puHs6pHlImOSMOUyLMPsCgp7ur6t4+9d+z/reMtZauZCPOJz2BW0m6ys5QusrOULrKzlC6ys5QusrOULrKzlAWpGxjzPPGmFFjzBFjzLdv1KR+W8Vcb1JjjHGBQ8AXgDPAduBr1tr9N256v13iLeDcTcARa+0xAGPMvwMvAVdVdrHYYwcGBjCOwXPSRWX0VW56EsckUQRAvdkEIIiTzvdcCRx6rVLfAACDA0OYKABg4uI4AI22vDfGYIyOqa+dGVjb+ay3pweAKIpwfVFTsVDQ4+WYxFqstZTLZRqNenqZq8pClL0SOH3J+zPAox89yBjzMvAyQF9fP9/4+jfJ9RYZLhX0CBcAG4cANKo1muOTAGzd+yEA5ypVYiM3gCjR85L0+phCCYCNT38RgJde/ArepEztn//h7wHYeewEAL7n4+dycq4nPz/vys0KghYmJ/N6/KGHAbgwNcHA8iUAPHD3PQB4Ro8PI4Ig5JVXvjunsmBhyr7SnfwN2FlrXwFeAVixYoUt9PjUW02CQl4m4MQAxIkqk4ipegWAmVZLPkkSCoUiAPmi3JygLahvtVpEzRoA+3ZvB6BQ6uf+5YsA6OnrB6DUK6+FfB7PzCITwOjYgU1AUVwoCrI9k8NzFNF6Xqw/0/dzOMbBceYEtVxrXkddWc4At13yfhVw7lonuK7Lov5BvJkKrWYbgKFSLwA2lvfNWpPzZVF2MxQlWMehd0CUNzQ4CMBMdQKAaGIKL5Ef37wgJmPr6z/i9MhSAHKJrJiVq2WqfYU+cgqJqYvnAZioykpKsDhWxgwjuZlR1O6YjVgXlWPkn9hYrJm/z1tINLIduNMYs9YYkwP+EPjJAq73Wy/XjWxrbWSM+Rbwc8Twfs9au+9a5xjA9RxKPUUq5TIAY0EdgEFf7GizWqPaEPNh1PG5rktpWOzm2jvvBCDnifkhsPS7PgDTE2cAOHhkPycPHwEg8cT8rF53NwDr732QJX1DAJwfHQXg7V3vAtAKAlIf4uh5Xs4nitScqePOq9bCQGx2kqR+5NqyEDOCtXYLsGUh17iVZEHKvh6xSUIcRySxILPSEttoc2Jbp2t12mqr05DO5AsMLB4BYPWylQCUbAMAP4rJq12uxoLY3NAQ0cQ0AGerYv9PHpZFF9Vn6O/tk/9nUt/Q1OEcPFdU0p8XX9IoFAgV0U2da5ja6SgijkPsPJHdTdczlEyRHcURU9NTtMOQxIhtXNwjIVmzLEicqtZpa3SBhnsjt9/FEw9uAGBgSuzygR2/BqA9U8NLI05FWKtWY0ko0U2s8fhErQrA2ImjTBfThCW87BULoSZS+w7v1Tm3GB5ZAYCnUYmrK66dRLSCqBNCziWZm5EEKJX6yHvi1Agls6tduAhAvR2QuHIjhpavAeC5xzZzu8Zro3t3ANA8eQwAE8XE6bWtKMMmln4rSs47ct5iPaYRt6k05IyGJidpQhlbSxKLMzx+7jgAnucSeTKfRUuG5Vp9YmJcY0mTq/lI14xkKJki23Fcin0DDPT0klM0TY+PATBeEWcVWIfepZKAPPHokwDc35fj5K/+B4CzRw/JtQJxWj5g1YwoiPGsJdbr59Jahx4TY6lr4tJUZM8oOKcxtKx8lmgGE1vL+fOSq7XVTK1bsxaAVYsX09/bh+vOD7NdZGcomSLbGEPO90k6WIS2JjBTLXFoztAiNmx8HICNIxLKjW9/jTN7d8v3Wi8xGn4ZzCUVGUGexXZS7LSe4WrRySSWXkVoUe16SR1cyRrKeo1KInY9yvkYXSbTivA9DXGi8fp7uXPFMq5cJvpNyVTZ1lqiMCQKY1pa/JnQOkiYl9j3rgc2skmLSDN7JOI4tXM7UU0yTdd+tCw6+y51dBZDGgqnNSJf42fHJJBcXq4tpidiKOl/i1Xp03FEuVNSldfG1BQA+/ftpVqr0lAAzCVdM5KhZGtGdEADVKuC1JNVMR+fuu8RAJ5ffw+tPb8C4MgOqVk0ZqqzC/XyXgM4ptNPsIody2wTIM1CYzUd1lqtfgCe1GOs1rB9wAsFpUVtPpSAIW1KjMUyULkhc29Uyhw53KLd7iL7ppNsbTaWxEbYKGbiotSjk5KkG5/f8IBM6NSHHP9gKwCtujgitzSADQRpNpDP/B6xrrm+YZyc/AzfSCZYuTBB2ooYWLkagN6SZI3lsTNMj0sCVVwh363esAkAJ5ejcvoEAOOjuwBo1GrcdvvtANw+LDXyrTvfB2C6WiZqtmZ9wBzSRXaGkrHNNjiOQ71Z5eykeHR/laBleb+k76O736NZkTbX0vs2ArDy3k+TTEk35fj2NwDoXymJxd2bX6BvUNLncFw6Lztee5WwJDWX+zd/HoDBfllBU0cP8s5//RsA+ZFlAKx/7LMANIMIe9d9AOQKYtlHt20l1grghgfvByCYlDrO+8cOUmlW5xv5Ze8gc9ZhfLrGZE3MwaIk7arLwo9mZsgNihLue/p3ARhcuoQh7QPmrISKh4+eBKTWcYeWXQ/s0+XdqrH2YblRuaI4wWOnpdaxZv3dDL4loWVT48K2Gp1De95n8TpR9sgddwBw+oMPybsydnFAbuDKVVKYqrRmODJ2plOYmku6ZiRDyRTZSZJQbzSZmKoRhBKKRVr1C9XHuJ6Ln5Ombv8SySAP73yTXqOmQlfC9BlB6tFd77FupRx//tBBAKYmL7CuKGYpqItJmjgiLbDb7ryNvKK9rhkkWtvwXfC0bmLSeou+AygUxMn6eS39Lh0BJ+bg6KF5/f4usjOUbJsHUczkxCRnymUiraolCunIikMyxsFoXcJ3BYE5J8eRbW8D0JgUukJSl0SiNhOSGEFxWsMgTujQ6rRJYQPlp8QGR6/rpj2KXqlT37PpKUx1BoAjp08BEIYtQi2YW02UAqsNYCKWj4zg+/68fn/Gyo6YmpykEQQdYotJsz4tFHmOQ6D8j4mTUvi5e9OX+NTKuwDY9dPvA9Ccku58FEOiDspq0wGbYLXH6Xh6A7QWY02C487eFIAwlPi8PHaai++/A8C5vRJn26CBTQlEqmzfUzZWUge/yBW4SVeUrhnJUDJuHjgUe3oY7mlTaQvyPEU06VJ0HdqVCwDs3PKvAIx8+ikeePIZANY+LuXXs8ck9AuThFirHa5yTwx0kJ0W9tNSq3UgKchxYVWOSVriRM/v2crx7e8BEDUbOmuPqLMq5JPhNBt1XcZbVaJud/3mk0yRnc/nWLNmLXm/wO7Tku112LspAl0XvyhVtpElqwAoH9vHqTVSn1jSL9/l1D4HcURiZplTAF5iSdraNuuTsLCgiY+xfmclGA3zfA0nc2GI0fNMoivBmM4qMeqI83rN0kA/PdWZeRMr50S2MeY2Y8wbxpgDxph9xpg/08+HjTG/MMYc1teheY14C8t8kB0Bf2Gt3WmM6QPeN8b8Avhj4HVr7Xd0i8e3gb+85pUMuJ4ln3dw1IOneUUaUSQ9RdySIOeeJ58HIMhbfKX8hqfOAtKIBbBhgIkUeXoNL4mY0dDNPCKU8YceeUyuf/EczfFxnY7QlmOtdTjttFY420S2xuIn2rJTmnKakJX6+1nmgu92KuTXlDmVba0dA8b0/6ox5gBChH8J2KyH/QvwJnMoO4piLk5PUQ8DULaH1SUcqymwvkerKiXQQ4d2AnDfI0/gzYjT3LtLeCOhOjWncoGx3dsAqE+JEt0o4MIBOe6D/5afuGixFLzKR/cyMyZEeeuJsve9KuTb6sQ4scb4aUydFPP0LRfqWxo+pqTQhIh+3+DOsxD1sRykMWYN8DCwFViqNyK9ISNXOedlY8wOY8yORsfD35oybwdpjOkFfgj8ubV2prMvZQ65dOfByJIltjxTI4riTrM1VhMQI+HYwPAyzseCymNv/hiA8v53O6aiMi4IT5lUZuoku7b8AIBACfbYkKQipuHcu78EYExDzCRsQ0o3QzLBcwdkO0mSxJiU4aR7ahbfu4F162XLR3NCxp65IKbMzwcU8mbWFs4h80K2ETf8Q+D71tr/1I/HjTHL9fvlwIV5jXgLy5zINgLhfwIOWGv/7pKvfgJ8E/iOvv54ztFsgg2beJHwNwDqVbGzx87I6xcefYHGeQkLT3wgCJ88cgJjL6cfdFLkyFKbql72nSw6/V7pwFbBLDyTtG6Spu2atJCQaGVvcL0QOR/c/ByupvxH98tGuKBd0WvFxLncJSyYa8t8zMgTwDeAD40xu/Szv0KU/ANjzJ8Ap4A/mNeIt7DMJxr5NVdv/DzzcQYzjkuh2EvSCPDS9DwR5L33lnD5+p/5Eo/83tdlcposHN+5jUSbvx0QKSodpFIIUmTSSZOkoRuXn+YwG2mkPypRRk+SKzJwz0MAPPiUhJ353iKjByQqOjUuUYzvi/9Y5Dh65ZuQMmyMwc3lCNsxOV9aTcODyskYOwHAlp/9CPv0CwB85otfBcB14fB2Ce8ibaelcbrrmI7iU95IuqtBB5WXlEllZjeNpmXYWHkjvevu5f6nZC9laVDAcODANs6flbnFbTFX7UjOL5ZK5IJ4vs31bm0kS8mcDA+WKImIdd9juSbF+lCzs9qJg/z0VQnr4s0vArDp2T/CNTLV0W3CkkrNiqC0Q4lCP7yEJaXmJq1f2NnDIg0HS2ukVr7+sd+hb1Ac5PGTEg5eHD9MQffUuNptaDnanI5aNE1IciNDv67cGMm2U5NETNenqbVaGEeQHYWSdi8qKRcviKiNSwP1tbe03vy553n8WbXf6gwPbv1fOb/RxtFEJOlsk3ZnmfEq6XfWWiKtoeRXCV1h/ZPPAjA4VOT4qLCxLl4UZxi06hjNx0vK4O/Niz13PEsrCrDzRHa29LMkplWrECchfl6UMaBkmKW9kkF6rZDjY0JcnB6TDvov39xC7tkvA/DZZyVSSXSj6YnRgzi6WyDd+J+4Lo6jm0dTqrDuSTckxD3SHV+mmeFSfRDAnn3vcPK4dOGjtoChRYheiuEBmeNgUWoqJd/gNGLmWWHtmpEsJXMyfBwG+DlD0Re4+FZrHlrXKCROp0WWFvDbF87xxts/A8D9nIRmn3nuKwAs23i6Q/3NaxUvsgmRZoWBrvA0GIyjGjMtWTkFRfv+w9IK231kD+1W0JkrQDsOKar5KIWCzXpbmw25HD2+00X2zSgZIxvaUYLnGfKa7SUtKbue0X0qgz0DRE66JUPtbhhROyv7Hn/++n8AcHadODffGmJ1huk1g3qDcllCyrImQbEW+PN+QqK8j7w2BZozQl+erFRox2k7TObcjmJGPEl6HHWs9bqumiCilPOIu5Thm08y3k4dM1Wp4AYeBY0m8poopE8lmqoGFDS8czSR8fw8jtrxWJG6b5ckHWGjSUsfFOCq8WxWy0xPC7JjtfuuRhCDQz34ngw24KfhpiK8GXX2QaZpfjuytNpyXLke6HdK0nEstfws/3AuyZxYWWs2yVmfC0YmXtAZtFriwmphkzzyXWJEQW6hh/6C9CBzKcOpIfF5tTpDQ1tZXk5uSLXWpqo3Jd2SV/DlvDDwiJUJlYbijnrRRmiIUiplauYsVJvKrnLlpuTVuRfzHm5grviMsCtJ14xkKNf9XL/rGsyYi0AdmMhs0OuXxcx/nquttUvmOihTZQMYY3ZYazdmOuh1yP/HPLtmJEPpKjtD+SSU/conMOb1yA2fZ+Y2+1aWrhnJUDJT9s38rO1rMHX/2hhz1hizS/9eXNA4WZiRm/1Z28roWn4pUxf4MvBVoGat/ZsbMU5WyO48a9taGwDps7ZvCrHWjllrd+r/VSBl6t5QyUrZV3rW9g3/MTdCPsLUBfiWMWaPMeZ7CyX8Z6XseT1r+5OWjzJ1ge8CdwAbEI763y7k+lkp+2M/aztruRJT11o7bq2NrbTP/xExh9ctWSn7pn7W9tWYuiklWuX3gb0LGSeTevb1PGs7Y7kaU/drxpgNiMk7AfzpQgbpZpAZSjeDzFC6ys5QusrOULrKzlC6ys5QusrOULrKzlC6ys5Q/g8XkJpr9siGwAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f4e20409080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Data exploration visualization code goes here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "import matplotlib.pyplot as plt\n",
    "import random\n",
    "%matplotlib inline\n",
    "\n",
    "#randomly choosing an image data and visualize the image\n",
    "\n",
    "index = random.randint(0,len(X_train))\n",
    "image = X_train[index].squeeze()\n",
    "\n",
    "plt.figure(figsize=(1,1))\n",
    "plt.imshow(image)\n",
    "print(y_train[index])\n",
    "# Visualizations will be shown in the notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f3b4dd7d5c0>"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHFW9//H3h7ApBAImYshCwBuQRY0wIu5REMIiiwrC\nVYiKBvyBFxWXgCgIoriBFxc0Sn4sajCCQi5GMSKbz2VLICxhkYABhsQk7GEn4Xv/OGdIMemeVGWm\np6dnPq/n6We6Tp2qOl3d098+S51SRGBmZlbFWs0ugJmZtR4HDzMzq8zBw8zMKnPwMDOzyhw8zMys\nMgcPMzOrzMGjH5L0c0lf76F9jZb0lKRBeflKSZ/uiX3n/f1Z0sSe2l+F435L0sOS/r2G2y+QtFtP\nl6sn9eTnoLsk3S3p3T2d15rHwaPF5C+tZyUtk/S4pP+VdKSkl9/LiDgyIk4pua8uvwAj4oGI2DAi\nVvRA2U+S9OtO+98zIs7t7r4rlmMUcCywXUS8rk6ejST9SNIDOXjOz8tDe7Os3VH2c9CZpHn5NT8l\naYWk5wrLx69hWbaJiGt6Om8VktbL7+FD+f/nX5J+UHLbb0k6p6fL1MocPFrTByNiMLAFcBrwVeDs\nnj6IpLV7ep99xBbAIxGxpNZKSesClwPbAxOAjYB3AI8AO/dWIZslIrbPPxg2BK4Bju5Yjohvd87f\nQp+TE4A3ATuR3tP3A3ObWqJWFhF+tNADWADs1iltZ+AlYIe8fA7wrfx8KHAp8DjwKOnLYC3g/LzN\ns8BTwFeAMUAAhwMPAFcX0tbO+7sS+A5wA/AEcAmwaV43HmivVV7Sl/ALwIv5eLcU9vfp/Hwt0j/4\n/cAS4Dxg47yuoxwTc9keBr7WxXnaOG+/NO/vhLz/3fJrfimX45wa234aWAxsWOZ9yOf/2nyOFwE/\nAdbN6wSckV/PE8CthfdpL+AOYBnwEPClwv73IX2xPQ78L/Cmwrqv5vzLgLuBXeuUsfg5GA+0k2pc\nS3I5P1ni8/by+9Pp/FwNnEn6TJ0EjAWuIAXYh0mfr40L27QD4/PzbwHTgF/n13A7sOMa5m3L52kZ\ncAHwe+CkOq/lL6RAWO+1jgT+mD8z/wKOKrwXxc/unGZ/D/SFh2se/UBE3ED6h6vVTnxsXjcM2Aw4\nPm0Sh5K+hD8Y6Rfl9wrbvBfYFtijziEPAz4FbA4sJ32JrK6MfwG+DfwuH+/NNbJ9Ij/eB2wFbEj6\nIi56F7ANsCvwDUnb1jnkj0kBZKv8eg4jfVn+DdgTWJjL8Yka2+4G/CUinlrd68pWAF8gBeq357L9\nv7xud+A9wNbAEOCjpC9YSLXFIyLVIncA/g4gaUdgKnAE8BrgF8CM3OyyDXA08Na83R6kQFbG60jn\nZATpB8JPJW1SctvO3gHcSfpcfZcUJL8FDAe2I533rvpb9icFmCHAn+n6M1Qzr6T1gIuBXwGbAhfl\nvPVcB3xZ0mcl7SBJHStyn96lwI2k8/OBnHfXiLgU+B7wm/yZ2amLYwwYDh79x0LSP1BnL5L+obeI\niBcj4prIP6e6cFJEPB0Rz9ZZf35E3B4RT5O+IA7q6FDvpo8Bp0fEffmL+zjg4E7NIt+MiGcj4hbg\nFmCVIJTL8lHguIhYFhELgB8Ch5Ysx2tIv8xLiYg5EXFdRCzPx/oFKWBBOv+DgTcAiog7I2JRYd12\nkjaKiMci4qac/hngFxFxfUSsiNQn9DywCylQrZe3WyciFkTEvSWL+iJwcv4czCT9it6m7Ovs5IGI\nOCuX79mI+GdEXB4RL0RqDjyjcA5quSoiLovUl3Y+MG4N8r4TeCkifpJf0++BOV3s51vAD0ifgzlA\nu6SP53W7ABtFxLfza5hPCu4Hd30aBi4Hj/5jBKkJobPvA/OBv0q6T9LkEvt6sML6+4F1SL+6u2vz\nvL/ivtcm1Zg6FEdHPUOqnXQ2FFi3xr5GlCzHI6SAW4qkrSVdKunfkp4k1bCGAkTE30m1p58CiyVN\nkbRR3vTDpKar+yVdJentOX0L4Ng8IOJxSY8Do4DN85fa50lNRUskXSBp87KvKyKWF5brnb8yXvEZ\nkfQ6SdNzZ/STpCazrj4Tnd/HDdYg7+akWnXdchXl4P7jiHgHqRbzPeAcSVuTzvnoTuf8K6TamtXg\n4NEPSHor6YvxH53X5V/ex0bEVsAHgS9K2rVjdZ1drq5mMqrwfDTpF+3DwNPAqwvlGkRq1ii734Wk\nf+LivpeT+h+qeDiXqfO+Hiq5/d+APSR19YVWdBZwFzA2IjYiNQ2+3CQSEWfmpo7tSc1XX87pN0bE\nfsBrSc0v0/MmDwKnRsSQwuPVETEtb/fbiHhXfn1BajbqbZ3fy++SakdvzOfgExTOQYMsIvVTFI2q\nlbGzXFv6b1Lta1vSOb+n0zkfHBEf7Nikx0rdTzh4tLA8nHQfUkfhryPithp59pH0H7l990lSs0fH\nsNvFpLbpqj4uaTtJrwZOBi7MTQr/BNaXtLekdUid1OsVtlsMjCkOK+5kGvAFSVtK2pCVfSTL6+Sv\nKZdlOnCqpMGStgC+SOp0LeN80pfJRZLeIGktSa+RdLykvWrkH0w6t09JegPw2Y4Vkt4q6W35fDwN\nPAeskLSupI9J2jgiXmTlewPwS+DIvJ0kbZDP6WBJ20h6f27vf47U+d/tYdQ9YDDp9T2Rh0J/qReO\n+Q9gUO7DWFvSh0kjqWqS9AVJ75H0qpz/U8D6pA73a4EXJB0raX1JgyS9UVLH/jo+u40OiC3DwaM1\n/Y+kZaQvuK8BpwOfrJN3LOmX9FOkf5CfRcSVed13gBNyNb3KP/v5pGaJf5P++f4LICKeIHUU/4r0\nK/9pXtms8Pv89xFJN7GqqXnfV5NGuzwHfK5CuYo+l49/H+lL5rd5/6sVEc+TOs3vAmaRvthvIDXD\nXF9jky8B/0ka8fNL4HeFdRvltMdITWePkNrdIbW9L8jNPEcCH8/Hn03q9/hJ3m4+6Zc8pGB8Gql2\n9W9SrWWNrr3oYSeSRp09AcwgdV43VH6fDiCdu8eAg4CZpBpQLc8BPyIFgodJAxI+FBH35x8oe5Fe\nw4K8/hek9w/Se7ou8KikGxrxelqNVt93ambWGiTNAX4UEec3uyz9nWseZtayJI2XtFluhjqcNKrt\nr80u10DQKleGmpnVsi2pSWkD4F7gwxFRdYCFrQE3W5mZWWVutjIzs8r6bbPV0KFDY8yYMc0uhplZ\ny5gzZ87DETFs9Tn7cfAYM2YMs2fPbnYxzMxahqT7V58rcbOVmZlV5uBhZmaVOXiYmVllDh5mZlaZ\ng4eZmVXm4GFmZpU5eJiZWWUNCx6SRkm6QtKdkuZJOianbypplqR78t9NcroknSlpvqRble7j3LGv\niTn/PZImNqrMZmZWTiNrHsuBYyNiW9L9gY+StB0wGbg8IsYCl+dlgD1J954YC0wi3Z0NSZuS7hXw\nNtJc+yd2BBwzM2uOhl1hHhGLSLeJJCKWSbqTdKvU/YDxOdu5wJXAV3P6eZFmarxO0hBJw3PeWRHx\nKICkWcAE0l3n+pQxk/9UM33BaXv3cknMzBqrV/o8JI0B3kK6C9tmObB0BJjX5mwjeOXN69tzWr30\nWseZJGm2pNlLly7tyZdgZmYFDQ8e+V7UFwGfj4gnu8paIy26SF81MWJKRLRFRNuwYaXm9jIzszXQ\n0IkRJa1DChy/iYg/5OTFkoZHxKLcLLUkp7cDowqbjwQW5vTxndKvbGS5u+KmKTOzxo62EnA2cGdE\nnF5YNQPoGDE1EbikkH5YHnW1C/BEbta6DNhd0ia5o3z3nGZmZk3SyJrHO4FDgdskzc1pxwOnAdPz\n/YYfAA7M62YCewHzgWeATwJExKOSTgFuzPlO7ug8NzOz5mjkaKt/ULu/AmDXGvkDOKrOvqYCU3uu\ndGZm1h2+wtzMzCpz8DAzs8r67W1orXV4BJtZ63HNw8zMKnPwMDOzytxsZb3CTVNm/YtrHmZmVpmD\nh5mZVeZmKzOzHjDQmmZd8zAzs8ocPMzMrDIHDzMzq8zBw8zMKnPwMDOzyjzayioZaCNKzKw21zzM\nzKyyRt6GdqqkJZJuL6T9TtLc/FjQcYdBSWMkPVtY9/PCNjtJuk3SfEln5tvbmplZEzWy2eoc4CfA\neR0JEfHRjueSfgg8Uch/b0SMq7Gfs4BJwHWkW9VOAP7cgPI2jZuCzKzVNKzmERFXAzXvNZ5rDwcB\n07rah6ThwEYRcW2+Te15wP49XVYzM6umWX0e7wYWR8Q9hbQtJd0s6SpJ785pI4D2Qp72nGZmZk3U\nrNFWh/DKWsciYHREPCJpJ+BiSdsDtfo3ot5OJU0iNXExevToHiyumZkV9XrNQ9LawIeA33WkRcTz\nEfFIfj4HuBfYmlTTGFnYfCSwsN6+I2JKRLRFRNuwYcMaUXwzM6M5zVa7AXdFxMvNUZKGSRqUn28F\njAXui4hFwDJJu+R+ksOAS5pQZjMzK2jkUN1pwLXANpLaJR2eVx3Mqh3l7wFulXQLcCFwZER0dLZ/\nFvgVMJ9UI+lXI63MzFpRw/o8IuKQOumfqJF2EXBRnfyzgR16tHBmZtYtvsLczMwqc/AwM7PKHDzM\nzKwyBw8zM6vMwcPMzCrz/Tz6OE+aaGZ9kWseZmZWmYOHmZlV5uBhZmaVOXiYmVllDh5mZlaZg4eZ\nmVXm4GFmZpU5eJiZWWUOHmZmVpmDh5mZVebgYWZmlTXyNrRTJS2RdHsh7SRJD0mamx97FdYdJ2m+\npLsl7VFIn5DT5kua3KjymplZeY2seZwDTKiRfkZEjMuPmQCStiPd23z7vM3PJA2SNAj4KbAnsB1w\nSM5rZmZN1Mh7mF8taUzJ7PsBF0TE88C/JM0Hds7r5kfEfQCSLsh57+jh4pqZWQXN6PM4WtKtuVlr\nk5w2AniwkKc9p9VLr0nSJEmzJc1eunRpT5fbzMyy3g4eZwGvB8YBi4Af5nTVyBtdpNcUEVMioi0i\n2oYNG9bdspqZWR2rDR6SDpQ0OD8/QdIfJO24JgeLiMURsSIiXgJ+ycqmqXZgVCHrSGBhF+lmZtZE\nZWoeX4+IZZLeBewBnEuqQVQmaXhh8QCgYyTWDOBgSetJ2hIYC9wA3AiMlbSlpHVJneoz1uTYZmbW\nc8p0mK/If/cGzoqISySdtLqNJE0DxgNDJbUDJwLjJY0jNT0tAI4AiIh5kqaTOsKXA0dFxIq8n6OB\ny4BBwNSImFf61ZmZWUOUCR4PSfoFsBvwXUnrUaLGEhGH1Eg+u4v8pwKn1kifCcwsUU4zM+slZZqt\nDiL98p8QEY8DmwJfbmipzMysTytTg3gGWAK8KyctB+5pZKHMzKxvKzPa6kTgq8BxOWkd4NeNLJSZ\nmfVtZZqtDgD2BZ4GiIiFwOBGFsrMzPq2MsHjhYgI8sV5kjZobJHMzKyvKxM8pufRVkMkfQb4G+kC\nPzMzG6BWO1Q3In4g6QPAk8A2wDciYlbDS2ZmZn1WqVl1c7BwwDAzM6BE8JC0jFUnI3wCmA0c2zFd\nupmZDRxlah6nkyYj/C1pltuDgdcBdwNTSVOQmJnZAFImeEyIiLcVlqdIui4iTpZ0fKMKZmbdM2by\nn1ZJW3Da3k0oifVHZUZbvSTpIElr5cdBhXV1761hZmb9V5ng8THgUNIUJYvz849LehVwdAPLZmZm\nfVSZobr3AR+ss/ofPVsc6wvc3GFmq1NmtNX6wOHA9sD6HekR8akGlsvMzPqwMs1W55NGV+0BXEW6\nFeyyRhbKzMz6tjKjrf4jIg6UtF9EnCvpt6T7e1gf5+YnM2uUMjWPF/PfxyXtAGwMjFndRpKmSloi\n6fZC2vcl3SXpVkl/lDQkp4+R9Kykufnx88I2O0m6TdJ8SWdKUqVXaGZmPa5M8JgiaRPg68AM0n3G\nv1diu3OACZ3SZgE7RMSbgH+y8h4hAPdGxLj8OLKQfhYwCRibH533aWZmvazMaKtf5adXAVuV3XFE\nXC1pTKe0vxYWrwM+0tU+JA0HNoqIa/PyecD+wJ/LlqOvqNWEBM1rRnKTlpl1R5nRVkOAw0hNVS/n\nj4j/6uaxPwX8rrC8paSbSbP3nhAR1wAjgPZCnvacVq+sk0i1FEaPHt3N4pmZWT1lOsxnkmoJtwEv\n9cRBJX2NdC/03+SkRcDoiHhE0k7AxZK2J82l1Vndq9ojYgowBaCtrc1Xv5uZNUiZ4LF+RHyxpw4o\naSKwD7BrvkMhEfE88Hx+PkfSvcDWpJrGyMLmI0mTNJqZWROVus5D0mckDZe0acdjTQ4maQLwVWDf\niHimkD5M0qD8fCtSx/h9EbEIWCZplzzK6jDgkjU5tpmZ9ZwyNY8XgO8DX2Nlk1Gwms5zSdNI07UP\nldQOnEgaXbUeMCuPuL0uj6x6D3CypOXACuDIiHg07+qzpJFbryJ1lLdcZ7mZWX9TJnh8kXSh4MNV\ndhwRh9RIPrtO3ouAi+qsmw3sUOXYZmbWWGWareYBz6w2l5mZDRhlah4rgLmSriB3akOPDNU1M7MW\nVSZ4XJwfZl3qaxdCmlnjlLnC/NzeKIiZmbWOusFD0m10fUHemxpSIjMz6/O6qnns02ulMDOzllI3\neETE/b1ZEDMzax1lhuqamZm9goOHmZlVVjd4SLo8//1u7xXHzMxaQVcd5sMlvRfYV9IFdJoePSJu\namjJzMysz+oqeHwDmEyaBv30TusCeH+jCmVmZn1bV6OtLgQulPT1iDilF8tk1lS+RW//5/e4+8pc\nYX6KpH1J06YDXBkRlza2WGZm1petdrSVpO8AxwB35McxOc3MzAaoMhMj7g2Mi4iXACSdC9xMurGT\nmZkNQGWv8xhSeL5xIwpiZmato0zw+A5ws6Rzcq1jDvDtMjuXNFXSEkm3F9I2lTRL0j357yY5XZLO\nlDRf0q2SdixsMzHnv0fSxGov0czMetpqg0dETAN2Af6QH2+PiAtK7v8cYEKntMnA5RExFrg8LwPs\nCYzNj0nAWZCCDen+528DdgZO7Ag4ZmbWHKWarSJiUUTMiIhLIuLfZXceEVcDj3ZK3g/ouEfIucD+\nhfTzIrkOGCJpOLAHMCsiHo2Ix4BZrBqQzMysFzVjbqvNImIRpKAEvDanjwAeLORrz2n10lchaZKk\n2ZJmL126tMcLbmZmSZnRVr1FNdKii/RVEyOmAFMA2tra6t7IyswXiZl1T5c1D0lrFTu7e8ji3BxF\n/rskp7cDowr5RgILu0g3M7Mm6TJ45Gs7bpE0ugePOQPoGDE1EbikkH5YHnW1C/BEbta6DNhd0ia5\no3z3nGZmZk1SptlqODBP0g3A0x2JEbHv6jaUNA0YDwyV1E4aNXUaMF3S4cADwIE5+0xgL2A+8Azw\nyXycRyWdAtyY850cEZ074c3MrBeVCR7fXNOdR8QhdVbtWiNvAEfV2c9UYOqalsPMzHpWmYkRr5K0\nBTA2Iv4m6dXAoMYXzczM+qoyEyN+BrgQ+EVOGgFc3MhCmZlZ31bmOo+jgHcCTwJExD2svDbDzMwG\noDLB4/mIeKFjQdLa1LnOwszMBoYyweMqSccDr5L0AeD3wP80tlhmZtaXlQkek4GlwG3AEaQhtSc0\nslBmZta3lRlt9VKeiv16UnPV3XlYrZmZDVCrDR6S9gZ+DtxLmmdqS0lHRMSfG104MzPrm8pcJPhD\n4H0RMR9A0uuBPwEOHmZmA1SZPo8lHYEju4+VkxmamdkAVLfmIelD+ek8STOB6aQ+jwNZOc+UmZkN\nQF01W32w8Hwx8N78fCng28CamQ1gdYNHRHyyNwtiZmato8xoqy2BzwFjivnLTMluZmb9U5nRVhcD\nZ5OuKn+pscUxM7NWUCZ4PBcRZza8JGZm1jLKDNX9b0knSnq7pB07Hmt6QEnbSJpbeDwp6fOSTpL0\nUCF9r8I2x0maL+luSXus6bHNzKxnlKl5vBE4FHg/K5utIi9XFhF3A+MAJA0CHgL+SLrt7BkR8YNi\nfknbAQcD2wObA3+TtHVErFiT45uZWfeVCR4HAFsVp2XvQbsC90bE/ZLq5dkPuCAingf+JWk+sDNw\nbQPKY2ZmJZRptroFGNKg4x8MTCssHy3pVklTJXVcSzICeLCQpz2nmZlZk5QJHpsBd0m6TNKMjkd3\nDyxpXWBf0v1BAM4CXk9q0lpEmlML0mSMndWc1VfSJEmzJc1eunRpd4toZmZ1lGm2OrFBx94TuCki\nFgN0/AWQ9Evg0rzYDowqbDcSWFhrhxExBZgC0NbW5mnjzcwapMz9PK5q0LEPodBkJWl4RCzKiwcA\nt+fnM4DfSjqd1GE+FrihQWUyM7MSylxhvoyVzUTrAusAT0fERmt6UEmvBj5AujNhh+9JGpePtaBj\nXUTMkzQduANYDhzlkVZmZs1VpuYxuLgsaX/SaKc1FhHPAK/plHZoF/lPBU7tzjHNzKznlOkwf4WI\nuJg1vMbDzMz6hzLNVh8qLK4FtFFntJNZTxsz+U810xectncvl8Qaxe9xayoz2qp4X4/lpP6I/RpS\nGjMzawll+jx8Xw8zM3uFrm5D+40utouIOKUB5TEzsxbQVc3j6RppGwCHk0ZKOXiYmQ1QXd2GtmN6\nECQNBo4hzXx7ASunDjEzswGoyz4PSZsCXwQ+BpwL7BgRj/VGwczMrO/qqs/j+8CHSHNFvTEinuq1\nUpmZWZ/W1UWCx5LmkjoBWJjv+PekpGWSnuyd4pmZWV/UVZ9H5avPzcx6iy8ubC4HCDMzq8zBw8zM\nKnPwMDOzyhw8zMysMgcPMzOrrMysumZ9kkfbrLla587nzapoWs1D0gJJt0maK2l2TttU0ixJ9+S/\nm+R0STpT0nxJt0rasVnlNjOz5jdbvS8ixkVEW16eDFweEWOBy/MywJ7A2PyYBJzV6yU1M7OXNTt4\ndLYfaQ4t8t/9C+nnRXIdMETS8GYU0MzMmhs8AvirpDmSJuW0zSJiEUD++9qcPgJ4sLBte057BUmT\nJM2WNHvp0qUNLLqZ2cDWzA7zd0bEQkmvBWZJuquLvKqRtsp91CNiCmkiR9ra2nyfdTOzBmla8IiI\nhfnvEkl/BHYGFksaHhGLcrPUkpy9HRhV2HwksLBXC2xmHuFmL2tKs5WkDfINppC0AbA7cDswA5iY\ns00ELsnPZwCH5VFXuwBPdDRvmZlZ72tWzWMz4I+SOsrw24j4i6QbgemSDgceAA7M+WcCewHzgWdI\ndzQ0M7MmaUrwiIj7gDfXSH8E2LVGegBH9ULRzPoUNxNZX9XXhuqamVkLcPAwM7PKHDzMzKwyBw8z\nM6vMwcPMzCpz8DAzs8p8Pw8zazgPOe5/XPMwM7PKHDzMzKwyN1u1MDcFdM23Wh3Y/P43lmseZmZW\nmYOHmZlV5mYrsyZz82Pf4uauclzzMDOzyhw8zMysMjdbmfUQNz/ZmmjVz41rHmZmVlmvBw9JoyRd\nIelOSfMkHZPTT5L0kKS5+bFXYZvjJM2XdLekPXq7zGZm9krNaLZaDhwbETdJGgzMkTQrrzsjIn5Q\nzCxpO+BgYHtgc+BvkraOiBW9WmqzbmjVpgnrGf3x/e/1mkdELIqIm/LzZcCdwIguNtkPuCAino+I\nfwHzgZ0bX1IzM6unqX0eksYAbwGuz0lHS7pV0lRJm+S0EcCDhc3aqRNsJE2SNFvS7KVLlzao1GZm\n1rTRVpI2BC4CPh8RT0o6CzgFiPz3h8CnANXYPGrtMyKmAFMA2traauYx6y/6WlNIXyuPNVZTah6S\n1iEFjt9ExB8AImJxRKyIiJeAX7KyaaodGFXYfCSwsDfLa2Zmr9SM0VYCzgbujIjTC+nDC9kOAG7P\nz2cAB0taT9KWwFjght4qr5mZraoZzVbvBA4FbpM0N6cdDxwiaRypSWoBcARARMyTNB24gzRS6yiP\ntDJrnL40t9NAbwrr6vU3+9z0evCIiH9Qux9jZhfbnAqc2rBCmZlZJb7C3MzMKvPcVjX0pWq79S3N\nbiqw5vL7v5JrHmZmVpmDh5mZVebgYWZmlTl4mJlZZQ4eZmZWmYOHmZlV5uBhZmaVOXiYmVllDh5m\nZlaZg4eZmVXm4GFmZpU5eJiZWWUOHmZmVpmDh5mZVebgYWZmlbVM8JA0QdLdkuZLmtzs8piZDWQt\nETwkDQJ+CuwJbEe63/l2zS2VmdnA1RLBA9gZmB8R90XEC8AFwH5NLpOZ2YCliGh2GVZL0keACRHx\n6bx8KPC2iDi6U75JwKS8uA1wdw8cfijwcA/spz/yuanP56Y+n5v6mn1utoiIYWUytso9zFUjbZWo\nFxFTgCk9emBpdkS09eQ++wufm/p8burzuamvlc5NqzRbtQOjCssjgYVNKouZ2YDXKsHjRmCspC0l\nrQscDMxocpnMzAaslmi2iojlko4GLgMGAVMjYl4vHb5Hm8H6GZ+b+nxu6vO5qa9lzk1LdJibmVnf\n0irNVmZm1oc4eJiZWWUOHnV4OpRXkjRV0hJJtxfSNpU0S9I9+e8mzSxjM0gaJekKSXdKmifpmJzu\ncyOtL+kGSbfkc/PNnL6lpOvzufldHgQzIEkaJOlmSZfm5ZY5Nw4eNXg6lJrOASZ0SpsMXB4RY4HL\n8/JAsxw4NiK2BXYBjsqfFZ8beB54f0S8GRgHTJC0C/Bd4Ix8bh4DDm9iGZvtGODOwnLLnBsHj9o8\nHUonEXE18Gin5P2Ac/Pzc4H9e7VQfUBELIqIm/LzZaQvghH43BDJU3lxnfwI4P3AhTl9QJ4bAEkj\ngb2BX+Vl0ULnxsGjthHAg4Xl9pxmr7RZRCyC9CUKvLbJ5WkqSWOAtwDX43MDvNwsMxdYAswC7gUe\nj4jlOctA/t/6EfAV4KW8/Bpa6Nw4eNRWajoUsw6SNgQuAj4fEU82uzx9RUSsiIhxpFkhdga2rZWt\nd0vVfJL2AZZExJxico2sffbctMRFgk3g6VDKWSxpeEQskjSc9OtywJG0Dilw/CYi/pCTfW4KIuJx\nSVeS+oWGSFo7/8IeqP9b7wT2lbQXsD6wEakm0jLnxjWP2jwdSjkzgIn5+UTgkiaWpSlyO/XZwJ0R\ncXphlc+NNEzSkPz8VcBupD6hK4CP5GwD8txExHERMTIixpC+X/4eER+jhc6NrzCvI/8i+BErp0M5\ntclFaiq8ws2rAAACEUlEQVRJ04DxpCmjFwMnAhcD04HRwAPAgRHRuVO9X5P0LuAa4DZWtl0fT+r3\nGOjn5k2kTt9BpB+q0yPiZElbkQahbArcDHw8Ip5vXkmbS9J44EsRsU8rnRsHDzMzq8zNVmZmVpmD\nh5mZVebgYWZmlTl4mJlZZQ4eZmZWmYOHWTdJep2kCyTdK+kOSTMlbV2cgdisv/EV5mbdkC8S/CNw\nbkQcnNPGAZs1tWBmDeaah1n3vA94MSJ+3pEQEXMpTKwpaYykayTdlB/vyOnDJV0taa6k2yW9O08k\neE5evk3SF3r/JZmtnmseZt2zAzBnNXmWAB+IiOckjQWmAW3AfwKXRcSp+R4yrybd92JEROwA0DG9\nh1lf4+Bh1njrAD/JzVkrgK1z+o3A1Dyx4sURMVfSfcBWkn4M/An4a1NKbLYabrYy6555wE6ryfMF\n0nxgbybVONaFl2+w9R7gIeB8SYdFxGM535XAUeQbBZn1NQ4eZt3zd2A9SZ/pSJD0VmCLQp6NgUUR\n8RJwKGmiQCRtQbqnwy9JM/PuKGkosFZEXAR8Hdixd16GWTVutjLrhogISQcAP5I0GXgOWAB8vpDt\nZ8BFkg4kTbn9dE4fD3xZ0ovAU8BhpDvH/X9JHT/sjmv4izBbA55V18zMKnOzlZmZVebgYWZmlTl4\nmJlZZQ4eZmZWmYOHmZlV5uBhZmaVOXiYmVll/wc9Sfv9ksUjRQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f3b4de04978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "classes = [i for i in range(n_classes)]\n",
    "\n",
    "unique, counts = np.unique(y_train, return_counts=True) #counting total data in each unique class\n",
    "\n",
    "#Plotting Bar graph \n",
    "plt.bar(classes,counts)\n",
    "plt.xlabel('Class')\n",
    "plt.ylabel('Number of Images')\n",
    "plt.title('Distribution of Classes in Training Set')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of classes with low images : 21\n",
      "[0, 6, 16, 19, 20, 21, 22, 23, 24, 27, 28, 29, 30, 32, 34, 36, 37, 39, 40, 41, 42]\n"
     ]
    }
   ],
   "source": [
    "#counting classes which have image data less than 500\n",
    "image_class_low = [i for i, j in enumerate(counts) if j<500]\n",
    "print('Number of classes with low images :',len(image_class_low))\n",
    "print(image_class_low)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classes with low images:6060\n"
     ]
    }
   ],
   "source": [
    "#counting total number of image data whose class fall under low class list\n",
    "count = 0\n",
    "print('Classes with low images:',end = \"\")\n",
    "for i, j in enumerate(y_train):\n",
    "    if(j in image_class_low):\n",
    "        count += 1\n",
    "        \n",
    "print(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"Augmentation techniques\"\"\"\n",
    "\n",
    "#changing brightness\n",
    "def augment_brightness_camera_images(image):\n",
    "    image1 = cv2.cvtColor(image,cv2.COLOR_RGB2HSV)\n",
    "    random_bright = .25+np.random.uniform()\n",
    "    #print(random_bright)\n",
    "    image1[:,:,2] = image1[:,:,2]*random_bright\n",
    "    image1 = cv2.cvtColor(image1,cv2.COLOR_HSV2RGB)\n",
    "    return image1\n",
    "\n",
    "#rotating images\n",
    "def rotate_images(img):\n",
    "    ang_range = 20\n",
    "    ang_rot = np.random.uniform(ang_range)-ang_range/2\n",
    "    rows,cols,ch = img.shape    \n",
    "    Rot_M = cv2.getRotationMatrix2D((cols/2,rows/2),ang_rot,1)\n",
    "    img = cv2.warpAffine(img,Rot_M,(cols,rows))\n",
    "    return img\n",
    "\n",
    "#translation\n",
    "def translate_images(img):\n",
    "    trans_range = 5\n",
    "    rows,cols,ch = img.shape\n",
    "    tr_x = trans_range*np.random.uniform()-trans_range/2\n",
    "    tr_y = trans_range*np.random.uniform()-trans_range/2\n",
    "    Trans_M = np.float32([[1,0,tr_x],[0,1,tr_y]])\n",
    "    img = cv2.warpAffine(img,Trans_M,(cols,rows))\n",
    "    return img\n",
    "\n",
    "#shear\n",
    "def shear_images(img):\n",
    "    shear_range = 10\n",
    "    rows,cols,ch = img.shape\n",
    "    pts1 = np.float32([[5,5],[20,5],[5,20]])\n",
    "    pt1 = 5+shear_range*np.random.uniform()-shear_range/2\n",
    "    pt2 = 20+shear_range*np.random.uniform()-shear_range/2\n",
    "    pts2 = np.float32([[pt1,5],[pt2,pt1],[5,pt2]])\n",
    "    shear_M = cv2.getAffineTransform(pts1,pts2)\n",
    "    img = cv2.warpAffine(img,shear_M,(cols,rows))\n",
    "    return img\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "k = 0\n",
    "X_train_extended = np.zeros((4*count, 32, 32, 3))  #initializing the array which will contain augmented images\n",
    "y_train_extended = np.zeros(4*count, dtype='int')  #initializing the array which will contain corresponding labels\n",
    "for i, j in enumerate(y_train):\n",
    "    if(j in image_class_low):\n",
    "        \n",
    "        X_train_extended[k] = augment_brightness_camera_images(X_train[i])\n",
    "        y_train_extended[k] = y_train[i]\n",
    "        k += 1\n",
    "        \n",
    "        X_train_extended[k] = rotate_images(X_train[i])\n",
    "        y_train_extended[k] = y_train[i]\n",
    "        k += 1\n",
    "        \n",
    "        X_train_extended[k] = translate_images(X_train[i])\n",
    "        y_train_extended[k] = y_train[i]\n",
    "        k += 1\n",
    "        \n",
    "        X_train_extended[k] = shear_images(X_train[i])\n",
    "        y_train_extended[k] = y_train[i]\n",
    "        k += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(59039, 32, 32, 3)\n",
      "(59039,)\n"
     ]
    }
   ],
   "source": [
    "#appending the augmented images to the training set\n",
    "X_train = np.append(X_train, X_train_extended, axis=0) \n",
    "y_train = np.append(y_train, y_train_extended, axis=0)\n",
    "print(X_train.shape)\n",
    "print(y_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f3b4dc7fb70>"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8XfO9//HXW8yEUKGRiNCGUlrVFL/qkKLETFt+1FSl\nOnBLq62hlBalt6qqWqWaK7SGlCLXdYuqGH73V0M0hlRVqkEkEkMQs/C5f3y/W1ZO9j5nnayzzz77\nnPfz8TiPs/d3Dfuzv3ut9Vnf75oUEZiZmS2ppVodgJmZtTcnEjMzq8SJxMzMKnEiMTOzSpxIzMys\nEicSMzOrZEAkEkm/knRiD81rpKSXJA3K7ydLOrQn5p3n99+SDuqp+XXjc0+V9Iykp5Zw+hmStuvp\nuHpSTy4HVUl6WNLHe3rcdiLpDEnPSprZ6lj6O0kXSjq+aR8QEW39B8wAXgXmA88D/wN8BVhqCee1\nXTenmQwcuoSxnwz8tg/U4Tq5DtfsZJxVgLOBx4GXgOn5/RpLWnft8gdMy9/5JeAt4LXC++NbHV+F\n77Vc/g2fzOvPv4AzS057KnBRhc9eD3ilsPy8N22OWl8vfe0P+C1wcjfGPxSY3Jsx9pcWya4RMRhY\nFzgDOAb4TU9/iKSle3qefcS6wLMRMbfeQEnLAjcD7wfGkZLKR4FngS16K8hWiYj3R8TKEbEycDtw\nRO19RPyw4/httJycAHwA+DDpN90GmNpLn70uMDcinumJmbVRnfdPrc62PZCtZ9BhT5i0cXsb2CS/\nvwg4Nb9eA7iO1Hp5jrRhWAq4JE/zKmlP8zvAKCCAQ0h74rcVypbO85sMnA7cBbwAXAusnoeNBWbW\ni5e0QX4DeDN/3n2F+R2aXy9FWtkfA+YCFwOr5mG1OA7KsT0DfLeTelo1T/90nt8Jef7b5e/8do7j\nogZ7OHOAlcv8Drn+/3+u49nAucCyeZiAn+bv8wJwf+F32gn4G2nv+EngW4X570LayNVanR8oDDuG\nhXvVDwPbNoixuByMBWYCR+dYZgMHl1je3vl9OtTPbcA5pGXqZGA0cAsp2T5DWr5WLUwzExibX58K\nXEba85wPPAhsvoTjjsn1NB+4HPg9DfZmgT+SkmKj7zoCuDovM/8CDi/8FsVld0qD6U8AHs2xTAN2\ny+XjOixzFwKzSMtzraX3kULd/h2YB/w3sE4uXzqP/zVS63h6nc9fCrgSeCovN5OBjQrD7wC+0OF3\nnFx4vyPwD9Jy+nPg/9XGz+Pemn/z53MMW5K2FU+Q1pf9C/NaHjirMOyXwPJ52Hak9ec7ua5nAQfm\nYV/L9fxGrperu6jbTUkt5rfy+M/k8kVaNaRem+mk5fMaYFiHev1yHj4POKfL9aKnNuit+qNBlwpp\n4/rVOhuQ04FfAcvkv48DqjcvFm6sLwZWAlagfiJ5Etgkj3MVubuKThJJfn0yHbq2WDSRfDH/mOsD\nKwN/AC7pENuvc1wfBF6nsKJ0mO/FpCQ3OE/7D+CQRnF2mPZyYELZ34G0h7tVXihHAQ8BR+VhOwBT\ngCGkpLJRYSGeDXw8v16NvIEENidt7LcEBpGS5wxS18yGpJVz7UK9vKdBjMXlYCywAPhBXg52InW1\nrNbF93zn9ymUHZrn9dUc3wrABsC2wLLAmqSN0JmFaTomh1dz3QwCfgzc0d1xc33MBI7I32kv0kao\nUSI5mbRT8VXS8qvCsEGkhHR8/g7vzXW+bSGOi7qoq72BYaQN+udJG7a18rDtgBmFcRfr2gI+R9ox\n2DAvSycDt+dhtQ3eH/OyskKdz18K+AJpmV+etENzT2F4w0SSf7P5wO65Lr+Z67KYSBYAB+S6OiPX\n5Tn5d9iJlIBWzOOfS0rKq5Faf9cDpxTqYgFwUv6s3YCXgVXy8MW6trqo28W6torzALYnrU+b5Xr5\nJfDnDvV6LWnncxRp56jTbuv+0rVVzyxg9Trlb5J+gHUj4s2IuD1yDXbi5Ih4OSJebTD8koh4MCJe\nBk4E9q4djK9oP+CsiHg0Il4CjgP26dCM/35EvBoR9wH3kRLKInIs/xc4LiLmR8QM4CeklaCMd5E2\n8qVExJSI+EtELMifdT7wyTz4TdKK/T7ShuuhiJhdGLaxpFUiYl5E3JvLvwScHxF3RsRbETGBlDS3\nIu15LZenWyYiZkTEP0uG+ibwg7wcXE9aGTcs+z07eDwizsvxvRoR/4iImyPijUhdhj8t1EE9t0bE\nDRHxFqn1stkSjLs18HZEnJu/0+9JSbuRU4EzScvBFGCmpP3zsK1IG7If5u8wndRdvE/n1bBQREyM\niNkR8XZEXEpKRGPKTk/aK/5hRDwcEQtyvFtIGl4Y54d5WVls3cyfe1Fe5l8jJaIPS1qpxGfvAkyN\niGsj4k3S79exG+6RiLgk/w5XACNJ6+PreXkCWF/SUqSN+1E51hdJO7TFunyNtJPzZkRMIi3fGzQK\nrmLd7gdcGBFTc70cC3xS0ojCOKdHxAt5/Z1M58tjv04kw0mZtKMfk/byb5T0qKRjS8zriW4Mf4y0\nV7FGqSg7t3aeX3HeSwNrFcqKZ1m9Qmq5dLQGaa+y47yG1xm3nmdJybcUSRtIuk7SU5JeBH6YYyAi\n/kzaO/sFMEfSBZJWyZN+lrQn95ikWyX9n1y+LnC0pOdrf6QTBNbOG7ijSBuJuZIul7R22e+VN1A1\njeqvjEWWEUnvljRR0pO5Di6i82Wi4+/Y2cau0bhrk1okDeMqyon+5xHxUVIL8d+BiyRtQKrzkR3q\n/DvAuzuJaxGSviDpvsL076N768W6wC8K0z9D6g4rbvAafj9JgyT9e17PXySt95SMYe3ivPPOZse6\nnVN4/SrwVkQ826FsZVKdLQcU6+I6Uqun5pmckGo6XRYr1u0i25Wc2Oax6PagzHblHf0ykUj6CKlS\n7ug4LO+dHB0R6wO7At+UtG1tcINZdtViWafweiRpT/cZUvN0xUJcg4Ch3ZjvLNLKVJz3AhZdgMt4\nJsfUcV5Plpz+T8AOJffkAM4j9WuPjohVSN0jqg2MiHMi4sOkg/cbAN/O5XdHxO6kFewaYGKe5Ang\ntIgYUvhbMSIuy9NdGhEfy98vgB+VjLMndfwtf0Taq9w018EXKNRBk8xm0Y0sLLpsNpRbUT8jtco2\nItX5Ix3qfHBE7FqbpLP5SVqftBx8FXhXRAwhLRON6qDe/J4gdb8WY1ghIu7sYrqaA0k7JtuQumne\nWwsv/19k/WTRJLlIXUoS5Xe8OppDOsaxYeF7rBoRq5acfpHvWKJuu7VdkTSY1OVWdnuwmH6VSCSt\nImkXUp/+byPigTrj7CLpvXnBeJHUNVLbE5hDOh7RXftL2ljSiqQ+9yvz3sU/gOUl7SxpGdIBsuUK\n080BRuWmbz2XAd+QtJ6klUl79ld02IvuUo5lInCapMGS1iX1+f625CwuIa3UV0l6n6SlJL1L0vGS\ndqoz/mBS3b4k6X2kBR5ISV7Slrk+XiYfGJS0rKT9JK2auxJqvw2k40BfydNJ0kq5TgdL2lDSNpKW\ny/N6tTBdKw0mfb8XJK0DfKsXPvMOYJCkr0paWtJnScer6pL0DUmfkLRCHv+LpD7zqaSTJd6QdLSk\n5fPe/aaSavOrLbuNEsPKpA3a0+mjdChpr7mRuUDkjWTNr4DvStooxztE0ue6rIWFBpOS+bOkhHFa\nh+FTgc/m778B6ZhkzXXA5pJ2zV3JR7LoTmBpef27EDhb0tC8DI+QtH3JWXTcLnVVt3OAEXkdq+cy\n4BBJH8jrzemkY09LfD1Pf0kk/ylpPmlj913S2REHNxh3NGkP+yXSyvLLiJich50OnJCbi91Z8S8h\ndV08RVoRvw4QES+Qzrq4kJTtX2bR5vHv8/9nJd3L4sbned9GOmvmNeDfuhFX0b/lz3+UtMG5NM+/\nSxHxOumA4N+Bm0gb+btITek760zyLdIBwPmkJHBFYdgquWweqXn9LKmfHlJf/YzcDfEVYP/8+feQ\njpOcm6ebTtrDh5SYzyC1up4itWaad+FVeSeRzl57AZhEOgmjqfLvtCep7uaRDsheT9qY1vMa6TqS\nOaT6+zLwmYh4LO+s7ET6DjPy8PNJvx+k33RZ4DlJd9WJ5X7Sgee7SHv376P+slIbfz5p/bszr39j\n8jGes4Df52XiftJJBmX9B2nvexbpzKb/6TD8TNIGeS5pXXhnxyoi5pCOK55FWkbfA/yVxnXZlaNJ\ny3vt7M4bSduiMi4EPihpnqQrS9TtTcAjpK7jxS4wjog/knZ4r87TjyQdN1litbOVzKwfkjQFODsi\nLml1LO0sd0vPAj4XEbe3Op6+pr+0SMwMkDRW0lq5q+oQ0t7qja2Oqx1JGidp1dz9cyLp+ORirS9L\nZwCZWf+xEanbaSXgn8BnczeNdd/HgN+RuvCmAXvk7kPrwF1bZmZWSdO6tiStI+kWSQ9JmibpyFx+\nstK59VPz306FaY6TNF3pbqc7FMrH5bLpKnfdh5mZ9ZKmtUgkDSPd+uLefJ7yFGAP0pkkL0XEmR3G\n35h0WtoWpAtm/sTCKzv/AXyadMbT3cC+EfG3Rp+9xhprxKhRo3r2C5mZ9XNTpkx5JiK6fZpz046R\nRLrtxez8er6kh+j8gp7dgctzH+S/JE1n4Z1lp0fEowCSLs/jNkwko0aN4p577umBb2FmNnBIeqzr\nsRbXK2dtSRoFfIiF5zofIel+SeMlrZbLhrPo7Q5m5rJG5R0/4zBJ90i65+mnn+7hb2BmZo00PZHk\nK7KvIt2w7EXSpf3vId0EbDbp5oFQ/9YJ0Un5ogURF0TEmIgYM3ToEl2AamZmS6Cpp//mS/SvAn4X\nEX+Ad64YrQ3/NelWBJBaGsX7Ao0gXQBEJ+VmZtZizTxrS6TbTj8UEWcVyot3kd2T9GAeSLeR2EfS\ncpLWI90+4C7SwfXR+X5Ty5JuvTypWXGbmVn3NLNFsjXp3kkPSKo9vvN4YF9Jm5G6p2aQ7u9DREyT\nNJF0EH0B6WlsbwFIOgK4gfQAmfERMa2JcZuZWTf0ywsSx4wZEz5ry8yseyRNiYjuPHwM8L22zMys\nIicSMzOrxInEzMwq8d1/zay0Ucf+12JlM87YuQWRWF/iFomZmVXiRGJmZpU4kZiZWSVOJGZmVokT\niZmZVeJEYmZmlTiRmJlZJU4kZmZWiROJmZlV4kRiZmaVOJGYmVklTiRmZlaJb9po1ofVu0ki+EaJ\n1re4RWJmZpU4kZiZWSVOJGZmVokTiZmZVeJEYmZmlTiRmJlZJU4kZmZWiROJmZlV4kRiZmaVOJGY\nmVklTiRmZlaJE4mZmVXiRGJmZpU4kZiZWSVOJGZmVokTiZmZVeJEYmZmlTQtkUhaR9Itkh6SNE3S\nkbl8dUk3SXok/18tl0vSOZKmS7pf0uaFeR2Ux39E0kHNitnMzLqvmS2SBcDREbERsBVwuKSNgWOB\nmyNiNHBzfg+wIzA6/x0GnAcp8QAnAVsCWwAn1ZKPmZm1XtOe2R4Rs4HZ+fV8SQ8Bw4HdgbF5tAnA\nZOCYXH5xRATwF0lDJA3L494UEc8BSLoJGAdc1qzYl5Sfr21mA1GvHCORNAr4EHAnsFZOMrVks2Ye\nbTjwRGGymbmsUXnHzzhM0j2S7nn66ad7+iuYmVkDTU8kklYGrgKOiogXOxu1Tll0Ur5oQcQFETEm\nIsYMHTp0yYI1M7Nua1rXFoCkZUhJ5HcR8YdcPEfSsIiYnbuu5ubymcA6hclHALNy+dgO5ZObGXdn\n6nVfuevKzAayZp61JeA3wEMRcVZh0CSgdubVQcC1hfID89lbWwEv5K6vG4DtJa2WD7Jvn8vMzKwP\naGaLZGvgAOABSVNz2fHAGcBESYcAjwN75WHXAzsB04FXgIMBIuI5SacAd+fxflA78G5mZq3XzLO2\n7qD+8Q2AbeuMH8DhDeY1Hhjfc9GZWW/yGY39m69sNzOzSpp6sN2sDO+tmrU3t0jMzKwSJxIzM6vE\nXVvWK9x9ZdZ/uUViZmaVuEViZtYDBnKr2y0SMzOrxInEzMwqcSIxM7NKnEjMzKwSH2xvY604uNcu\nt9EfyAc+zb9/b3OLxMzMKnEiMTOzSrrs2pK0F/DHiJgv6QRgc+DUiLi36dENEG6Gm1k7K9MiOTEn\nkY8BOwATgPOaG5aZmbWLMonkrfx/Z+C8iLgWWLZ5IZmZWTspk0ielHQ+sDdwvaTlSk5nZmYDQJmE\nsDdwAzAuIp4HVge+3dSozMysbXSZSCLiFWAu8LFctAB4pJlBmZlZ++gykUg6CTgGOC4XLQP8tplB\nmZlZ+yjTtbUnsBvwMkBEzAIGNzMoMzNrH2USyRsREUAASFqpuSGZmVk7KZNIJuaztoZI+hLwJ+DX\nzQ3LzMzaRZdXtkfEmZI+DbwIbAh8LyJuanpkBviq94HAv3Hf0i43Ju1LSt39NycOJw8zM1tMmXtt\nzScfHyl4AbgHODoiHm1GYGZm1h7KtEjOAmYBlwIC9gHeDTwMjAfGNis4MzPr+8ocbB8XEedHxPyI\neDEiLgB2iogrgNWaHJ+ZmfVxZRLJ25L2lrRU/tu7MKxjl5eZmQ0wZRLJfsABpNukzMmv95e0AnBE\nE2MzM7M2UOb030eBXRsMvqNnwzEzs3ZT5qyt5YFDgPcDy9fKI+KLTYzLzMzaRJmurUtIZ2ntANwK\njADmNzMoMzNrH2USyXsj4kTg5YiYQHpS4qZdTSRpvKS5kh4slJ0s6UlJU/PfToVhx0maLulhSTsU\nysflsumSju3e1zMzs2Yrk0jezP+fl7QJsCowqsR0FwHj6pT/NCI2y3/XA0jamHR9yvvzNL+UNEjS\nIOAXwI7AxsC+eVwzM+sjylyQeIGk1YATgUnAysD3upooIm6TNKpkHLsDl0fE68C/JE0HtsjDpteu\nnpd0eR73byXna2ZmTVbmrK0L88tbgfV74DOPkHQgC2+xMg8YDvylMM7MXAbwRIfyLevNVNJhwGEA\nI0eO7IEwzbrHN1+0JdEflpsyT0gcIunrks6SdE7tbwk/7zzgPcBmwGzgJ7WPqTNudFK+eGHEBREx\nJiLGDB06dAnDMzOz7irTtXU9qbXwAPB2lQ+LiDm115J+DVyX384E1imMOoJ0fy86KTczsz6gTCJZ\nPiK+2RMfJmlYRMzOb/cEamd0TQIulXQWsDYwGriL1CIZLWk94EnSAfnP90QsZmbWM8okkkvykxGv\nA16vFUbEc51NJOky0p2B15A0EzgJGCtpM1L31Azgy3le0yRNJB1EXwAcHhFv5fkcAdwADALGR8S0\n7nxBMzNrrjKJ5A3gx8B3WXh8IujiwHtE7Fun+DedjH8acFqd8utJ3WtmZtYHlUkk3yRdlPhMs4Mx\nM7P2U+aCxGnAK80OxMzM2lOZFslbwFRJt7DoMZKvNy0qMzNrG2USyTX5z8zMbDFlrmyf0BuBmJlZ\ne2qYSCQ9QCeP0o2IDzQlIjMzayudtUh26bUozMysbTVMJBHxWG8GYtaf9faN+frDjQD7k/7+e5Q5\n/dfMzKwhJxIzM6uks4PtN0fEtpJ+FBHH9GZQ1lr1muH9pQluZj2vs4PtwyR9EtgtP5lwkWeDRMS9\nTY3MzMzaQmeJ5HvAsaRngJzVYVgA2zQrKKuuvx/cM7O+o7Oztq4ErpR0YkSc0osxmZlZGylzZfsp\nknYDPpGLJkfEdZ1NY2ZmA0eXiUTS6cAWwO9y0ZGSto6I45oaWT/T17qafEDdzHpKmZs27gxsFhFv\nA0iaAPwVcCIxM7PS15EMKbxetRmBmJlZeyrTIjkd+Gt+HolIx0rcGjEzM6DcwfbLJE0GPkJKJMdE\nxFPNDszMzNpDmRYJETEbmNTkWMysG9rhBA7wSRwDge+1ZWZmlTiRmJlZJZ12bUlaCrg/IjbppXis\njblrw2xg6rRFkq8duU/SyF6Kx8zM2kyZg+3DgGmS7gJerhVGxG5Ni8rMzNpGmUTy/aZHYWZmbavM\ndSS3SloXGB0Rf5K0IjCo+aGZmVk76PKsLUlfAq4Ezs9Fw4FrmhmUmZm1jzKn/x4ObA28CBARjwBr\nNjMoMzNrH2USyesR8UbtjaSlSU9INDMzK3Ww/VZJxwMrSPo08DXgP5sbllnr+Fkt/Z9/455VpkVy\nLPA08ADwZeB64IRmBmVmZu2jzFlbb+eHWd1J6tJ6OCLctWXWT/WlvfWBfreEzr5/X6qbMmdt7Qz8\nEzgHOBeYLmnHEtONlzRX0oOFstUl3STpkfx/tVwuSedImi7pfkmbF6Y5KI//iKSDluRLmplZ85Tp\n2voJ8KmIGBsRnwQ+Bfy0xHQXAeM6lB0L3BwRo4Gb83uAHYHR+e8w4DxIiQc4CdiS9Nz4k2rJx8zM\n+oYyiWRuREwvvH8UmNvVRBFxG/Bch+LdgQn59QRgj0L5xZH8BRgiaRiwA3BTRDwXEfOAm1g8OZmZ\nWQs1PEYi6TP55TRJ1wMTScdI9gLuXsLPWys/JIuImC2pdj3KcOCJwngzc1mjcjMz6yM6O9i+a+H1\nHOCT+fXTQE93L6lOWXRSvvgMpMNI3WKMHOmbFVvn+tIBZbN21zCRRMTBTfi8OZKG5dbIMBZ2kc0E\n1imMNwKYlcvHdiifXG/GEXEBcAHAmDFjfFaZmVkvKXPW1nqSzpL0B0mTan9L+HmTgNqZVwcB1xbK\nD8xnb20FvJC7wG4Atpe0Wj7Ivn0uMzOzPqLMle3XAL8hXc3+dtkZS7qM1JpYQ9JM0tlXZwATJR0C\nPE463gLpIsedgOnAK8DBABHxnKRTWHhM5gcR0fEAvpmZtVCZRPJaRJzT3RlHxL4NBm1bZ9wg3Ryy\n3nzGA+O7+/lmZtY7yiSSn0k6CbgReL1WGBH3Ni0qMzNrG2USyabAAcA2LOzaivzezMwGuDKJZE9g\n/eKt5M3MzGrKJJL7gCGUuJq9v/A1BtaIl42Bzb9/fWUSyVrA3yXdzaLHSHZrWlRmZtY2yiSSk5oe\nhZmZta0yzyO5tTcCMTOz9tRlIpE0n4X3t1oWWAZ4OSJWaWZgZmbWHsq0SAYX30vag/RsEDMzs1LP\nI1lERFyDryExM7OsTNfWZwpvlwLG0OBW7mZmNvCUOWur+FySBcAM0hMNzczMSh0jacZzSczMrJ/o\n7FG73+tkuoiIU5oQj5mZtZnOWiQv1ylbCTgEeBfgRGJmZp0+avcntdeSBgNHkh44dTnwk0bTmZnZ\nwNLpMRJJqwPfBPYDJgCbR8S83gjMzMzaQ2fHSH4MfAa4ANg0Il7qtajMzKxtdHZB4tHA2sAJwCxJ\nL+a/+ZJe7J3wzMysr+vsGEm3r3o3M7OBx8nCzMwqKXNlu1nL1HsiHfipdP2Jf+P25xaJmZlV4kRi\nZmaVOJGYmVklTiRmZlaJE4mZmVXiRGJmZpU4kZiZWSW+jsTM2pKvP+k73CIxM7NKnEjMzKwSJxIz\nM6vEicTMzCppycF2STOA+cBbwIKIGJOfxngFMAqYAewdEfMkCfgZsBPwCvCFiLi3FXFb3+KDrUuu\nXt253mxJtbJF8qmI2CwixuT3xwI3R8Ro4Ob8HmBHYHT+Oww4r9cjNTOzhvpS19bupOfCk//vUSi/\nOJK/AEMkDWtFgGZmtrhWJZIAbpQ0RdJhuWytiJgNkP+vmcuHA08Upp2Zy8zMrA9o1QWJW0fELElr\nAjdJ+nsn46pOWSw2UkpIhwGMHDmyZ6I0M7MutaRFEhGz8v+5wNXAFsCcWpdV/j83jz4TWKcw+Qhg\nVp15XhARYyJizNChQ5sZvpmZFfR6IpG0kqTBtdfA9sCDwCTgoDzaQcC1+fUk4EAlWwEv1LrAzMys\n9VrRtbUWcHU6q5elgUsj4o+S7gYmSjoEeBzYK49/PenU3+mk038P7v2QzcyskV5PJBHxKPDBOuXP\nAtvWKQ/g8F4IzczMlkBfOv3XzMzakBOJmZlV4kRiZmaVOJGYmVklTiRmZlaJE4mZmVXiRGJmZpU4\nkZiZWSVOJGZmVokTiZmZVeJEYmZmlTiRmJlZJU4kZmZWiROJmZlV4kRiZmaVOJGYmVklTiRmZlaJ\nE4mZmVXiRGJmZpU4kZiZWSVOJGZmVokTiZmZVeJEYmZmlTiRmJlZJU4kZmZWiROJmZlV4kRiZmaV\nOJGYmVklTiRmZlaJE4mZmVXiRGJmZpU4kZiZWSVOJGZmVokTiZmZVeJEYmZmlbRNIpE0TtLDkqZL\nOrbV8ZiZWdIWiUTSIOAXwI7AxsC+kjZubVRmZgZtkkiALYDpEfFoRLwBXA7s3uKYzMwMUES0OoYu\nSfocMC4iDs3vDwC2jIgjCuMcBhyW324IPNxDH78G8EwPzau/cd005rppzHXTWKvrZt2IGNrdiZZu\nRiRNoDpli2TAiLgAuKDHP1i6JyLG9PR8+wPXTWOum8ZcN421a920S9fWTGCdwvsRwKwWxWJmZgXt\nkkjuBkZLWk/SssA+wKQWx2RmZrRJ11ZELJB0BHADMAgYHxHTeunje7y7rB9x3TTmumnMddNYW9ZN\nWxxsNzOzvqtdurbMzKyPciIxM7NKnEga8C1ZFpI0XtJcSQ8WylaXdJOkR/L/1VoZY6tIWkfSLZIe\nkjRN0pG5fMDXj6TlJd0l6b5cN9/P5etJujPXzRX5BJoBSdIgSX+VdF1+35Z140RSh2/JspiLgHEd\nyo4Fbo6I0cDN+f1AtAA4OiI2ArYCDs/LiusHXge2iYgPApsB4yRtBfwI+Gmum3nAIS2MsdWOBB4q\nvG/LunEiqc+3ZCmIiNuA5zoU7w5MyK8nAHv0alB9RETMjoh78+v5pI3CcFw/RPJSfrtM/gtgG+DK\nXD4g6wZA0ghgZ+DC/F60ad04kdQ3HHii8H5mLrOF1oqI2ZA2psCaLY6n5SSNAj4E3InrB3in62Yq\nMBe4Cfgn8HxELMijDOR162zgO8Db+f27aNO6cSKpr8tbspgVSVoZuAo4KiJebHU8fUVEvBURm5Hu\nRrEFsFG90Xo3qtaTtAswNyKmFIvrjNoWddMWFyS2gG/J0rU5koZFxGxJw0h7nAOSpGVISeR3EfGH\nXOz6KYiI5yVNJh1HGiJp6bznPVDXra2B3STtBCwPrEJqobRl3bhFUp9vydK1ScBB+fVBwLUtjKVl\ncr/2b4Bd+W0zAAACU0lEQVSHIuKswqABXz+Shkoakl+vAGxHOoZ0C/C5PNqArJuIOC4iRkTEKNL2\n5c8RsR9tWje+sr2BvKdwNgtvyXJai0NqGUmXAWNJt7ieA5wEXANMBEYCjwN7RUTHA/L9nqSPAbcD\nD7Cwr/t40nGSAV0/kj5AOmA8iLTTOjEifiBpfdIJLKsDfwX2j4jXWxdpa0kaC3wrInZp17pxIjEz\ns0rctWVmZpU4kZiZWSVOJGZmVokTiZmZVeJEYmZmlTiRmFUk6d2SLpf0T0l/k3S9pA2Kd0s26898\nZbtZBfmCxKuBCRGxTy7bDFirpYGZ9SK3SMyq+RTwZkT8qlYQEVMp3PRT0ihJt0u6N/99NJcPk3Sb\npKmSHpT08XyTw4vy+wckfaP3v5JZ97hFYlbNJsCULsaZC3w6Il6TNBq4DBgDfB64ISJOy8/AWZH0\n3I7hEbEJQO0WI2Z9mROJWfMtA5ybu7zeAjbI5XcD4/NNH6+JiKmSHgXWl/Rz4L+AG1sSsVk3uGvL\nrJppwIe7GOcbpHuUfZDUElkW3nlg2CeAJ4FLJB0YEfPyeJOBw8kPPTLry5xIzKr5M7CcpC/VCiR9\nBFi3MM6qwOyIeBs4gHQTQyStS3omxa9JdxDeXNIawFIRcRVwIrB573wNsyXnri2zCiIiJO0JnC3p\nWOA1YAZwVGG0XwJXSdqLdJvwl3P5WODbkt4EXgIOJD0R7z8k1Xbyjmv6lzCryHf/NTOzSty1ZWZm\nlTiRmJlZJU4kZmZWiROJmZlV4kRiZmaVOJGYmVklTiRmZlbJ/wI0cTiapbQvyAAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f3b4dd677b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#visualize change in image data after augmentation\n",
    "\n",
    "classes = [i for i in range(n_classes)]\n",
    "\n",
    "unique, counts = np.unique(y_train, return_counts=True)\n",
    "\n",
    "\n",
    "plt.bar(classes,counts)\n",
    "plt.xlabel('Class')\n",
    "plt.ylabel('Number of Images')\n",
    "plt.title('Distribution of Classes in Training Set after augmentation')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "## Step 2: Design and Test a Model Architecture\n",
    "\n",
    "Design and implement a deep learning model that learns to recognize traffic signs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-process the Data Set (normalization, grayscale, etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Converting rgb image into single channel grayscale image and normalize image data using (X-128)/128"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.utils import shuffle\n",
    "\n",
    "X_train, y_train = shuffle(X_train, y_train)  #shuffling the data\n",
    "\n",
    "grayconver= np.array([[0.2989],[0.5870],[0.1140]]) #weights for each colour channel to form a single channel\n",
    "def gray_normalize(img):\n",
    "    img1 = np.zeros((len(img),32,32,1))\n",
    "    img1[:,:,:,] = np.dot(img[...,:3],grayconver)\n",
    "    #print(img1[0].shape)\n",
    "    img1 = (np.array(img1) - 128.0) / 128.0\n",
    "    return img1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#preprocessing training,validation and test data\n",
    "X_train = gray_normalize(X_train)\n",
    "X_valid = gray_normalize(X_valid)\n",
    "X_test = gray_normalize(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "EPOCHS = 80  \n",
    "BATCH_SIZE = 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Architecture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Lenet Architechture\"\"\"\n",
    "\n",
    "from tensorflow.contrib.layers import flatten\n",
    "\n",
    "def LeNet(x):    \n",
    "    # Arguments used for tf.truncated_normal, randomly defines variables for the weights and biases for each layer\n",
    "    mu = 0\n",
    "    sigma = 0.1\n",
    "    \n",
    "    # SOLUTION: Layer 1: Convolutional. Input = 32x32x1. Output = 28x28x6.\n",
    "    conv1_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 1, 6), mean = mu, stddev = sigma))\n",
    "    conv1_b = tf.Variable(tf.zeros(6))\n",
    "    conv1   = tf.nn.conv2d(x, conv1_W, strides=[1, 1, 1, 1], padding='VALID') + conv1_b\n",
    "\n",
    "    # SOLUTION: Activation.\n",
    "    conv1 = tf.nn.relu(conv1)\n",
    "\n",
    "    # SOLUTION: Pooling. Input = 28x28x6. Output = 14x14x6.\n",
    "    conv1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    # SOLUTION: Layer 2: Convolutional. Output = 10x10x16.\n",
    "    conv2_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 6, 16), mean = mu, stddev = sigma))\n",
    "    conv2_b = tf.Variable(tf.zeros(16))\n",
    "    conv2   = tf.nn.conv2d(conv1, conv2_W, strides=[1, 1, 1, 1], padding='VALID') + conv2_b\n",
    "    \n",
    "    # SOLUTION: Activation.\n",
    "    conv2 = tf.nn.relu(conv2)\n",
    "\n",
    "    # SOLUTION: Pooling. Input = 10x10x16. Output = 5x5x16.\n",
    "    conv2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')\n",
    "\n",
    "    # SOLUTION: Flatten. Input = 5x5x16. Output = 400.\n",
    "    fc0   = flatten(conv2)\n",
    "    #fc0   = tf.nn.dropout(fc0,keep_prob)\n",
    "    \n",
    "    # SOLUTION: Layer 3: Fully Connected. Input = 400. Output = 120.\n",
    "    fc1_W = tf.Variable(tf.truncated_normal(shape=(400, 120), mean = mu, stddev = sigma))\n",
    "    fc1_b = tf.Variable(tf.zeros(120))\n",
    "    fc1   = tf.matmul(fc0, fc1_W) + fc1_b\n",
    "    \n",
    "    # SOLUTION: Activation.\n",
    "    fc1    = tf.nn.relu(fc1)\n",
    "    #fc1    = tf.nn.dropout(fc1,keep_prob)\n",
    "\n",
    "    # SOLUTION: Layer 4: Fully Connected. Input = 120. Output = 84.\n",
    "    fc2_W  = tf.Variable(tf.truncated_normal(shape=(120, 84), mean = mu, stddev = sigma))\n",
    "    fc2_b  = tf.Variable(tf.zeros(84))\n",
    "    fc2    = tf.matmul(fc1, fc2_W) + fc2_b\n",
    "    \n",
    "    # SOLUTION: Activation.\n",
    "    fc2    = tf.nn.relu(fc2)\n",
    "    #fc2    = tf.nn.dropout(fc2,keep_prob)\n",
    "\n",
    "    # SOLUTION: Layer 5: Fully Connected. Input = 84. Output = 43.\n",
    "    fc3_W  = tf.Variable(tf.truncated_normal(shape=(84, 43), mean = mu, stddev = sigma))\n",
    "    fc3_b  = tf.Variable(tf.zeros(43))\n",
    "    logits = tf.matmul(fc2, fc3_W) + fc3_b\n",
    "    \n",
    "    return logits\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train, Validate and Test the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A validation set can be used to assess how well the model is performing. A low accuracy on the training and validation\n",
    "sets imply underfitting. A high accuracy on the training set but low accuracy on the validation set implies overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, (None, 32, 32, 1))\n",
    "y = tf.placeholder(tf.int32, (None))\n",
    "one_hot_y = tf.one_hot(y, 43)\n",
    "keep_prob = tf.placeholder(tf.float32) # probability to keep units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rate = 0.001 #learning rate\n",
    "\n",
    "logits = LeNet(x)\n",
    "\n",
    "#calculating loss operation and optimizer\n",
    "\n",
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=one_hot_y, logits=logits)\n",
    "loss_operation = tf.reduce_mean(cross_entropy)\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate = rate)\n",
    "training_operation = optimizer.minimize(loss_operation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# calculating validation accuracy\n",
    "\n",
    "correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_y, 1))\n",
    "accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "def evaluate(X_data, y_data):\n",
    "    num_examples = len(X_data)\n",
    "    total_accuracy = 0\n",
    "    sess = tf.get_default_session()\n",
    "    for offset in range(0, num_examples, BATCH_SIZE):\n",
    "        batch_x, batch_y = X_data[offset:offset+BATCH_SIZE], y_data[offset:offset+BATCH_SIZE]\n",
    "        accuracy = sess.run(accuracy_operation, feed_dict={x: batch_x, y: batch_y, keep_prob: 0.5})\n",
    "        total_accuracy += (accuracy * len(batch_x))\n",
    "    return total_accuracy / num_examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "\n",
      "EPOCH 1 ...\n",
      "Validation Accuracy = 0.800\n",
      "\n",
      "EPOCH 2 ...\n",
      "Validation Accuracy = 0.884\n",
      "\n",
      "EPOCH 3 ...\n",
      "Validation Accuracy = 0.883\n",
      "\n",
      "EPOCH 4 ...\n",
      "Validation Accuracy = 0.917\n",
      "\n",
      "EPOCH 5 ...\n",
      "Validation Accuracy = 0.924\n",
      "\n",
      "EPOCH 6 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 7 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 8 ...\n",
      "Validation Accuracy = 0.933\n",
      "\n",
      "EPOCH 9 ...\n",
      "Validation Accuracy = 0.929\n",
      "\n",
      "EPOCH 10 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 11 ...\n",
      "Validation Accuracy = 0.933\n",
      "\n",
      "EPOCH 12 ...\n",
      "Validation Accuracy = 0.934\n",
      "\n",
      "EPOCH 13 ...\n",
      "Validation Accuracy = 0.932\n",
      "\n",
      "EPOCH 14 ...\n",
      "Validation Accuracy = 0.926\n",
      "\n",
      "EPOCH 15 ...\n",
      "Validation Accuracy = 0.920\n",
      "\n",
      "EPOCH 16 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 17 ...\n",
      "Validation Accuracy = 0.940\n",
      "\n",
      "EPOCH 18 ...\n",
      "Validation Accuracy = 0.948\n",
      "\n",
      "EPOCH 19 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 20 ...\n",
      "Validation Accuracy = 0.933\n",
      "\n",
      "EPOCH 21 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 22 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 23 ...\n",
      "Validation Accuracy = 0.940\n",
      "\n",
      "EPOCH 24 ...\n",
      "Validation Accuracy = 0.935\n",
      "\n",
      "EPOCH 25 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 26 ...\n",
      "Validation Accuracy = 0.943\n",
      "\n",
      "EPOCH 27 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 28 ...\n",
      "Validation Accuracy = 0.925\n",
      "\n",
      "EPOCH 29 ...\n",
      "Validation Accuracy = 0.946\n",
      "\n",
      "EPOCH 30 ...\n",
      "Validation Accuracy = 0.935\n",
      "\n",
      "EPOCH 31 ...\n",
      "Validation Accuracy = 0.946\n",
      "\n",
      "EPOCH 32 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 33 ...\n",
      "Validation Accuracy = 0.940\n",
      "\n",
      "EPOCH 34 ...\n",
      "Validation Accuracy = 0.938\n",
      "\n",
      "EPOCH 35 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 36 ...\n",
      "Validation Accuracy = 0.950\n",
      "\n",
      "EPOCH 37 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 38 ...\n",
      "Validation Accuracy = 0.934\n",
      "\n",
      "EPOCH 39 ...\n",
      "Validation Accuracy = 0.930\n",
      "\n",
      "EPOCH 40 ...\n",
      "Validation Accuracy = 0.932\n",
      "\n",
      "EPOCH 41 ...\n",
      "Validation Accuracy = 0.931\n",
      "\n",
      "EPOCH 42 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 43 ...\n",
      "Validation Accuracy = 0.936\n",
      "\n",
      "EPOCH 44 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 45 ...\n",
      "Validation Accuracy = 0.937\n",
      "\n",
      "EPOCH 46 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 47 ...\n",
      "Validation Accuracy = 0.942\n",
      "\n",
      "EPOCH 48 ...\n",
      "Validation Accuracy = 0.948\n",
      "\n",
      "EPOCH 49 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 50 ...\n",
      "Validation Accuracy = 0.943\n",
      "\n",
      "EPOCH 51 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 52 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 53 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 54 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 55 ...\n",
      "Validation Accuracy = 0.938\n",
      "\n",
      "EPOCH 56 ...\n",
      "Validation Accuracy = 0.947\n",
      "\n",
      "EPOCH 57 ...\n",
      "Validation Accuracy = 0.943\n",
      "\n",
      "EPOCH 58 ...\n",
      "Validation Accuracy = 0.947\n",
      "\n",
      "EPOCH 59 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 60 ...\n",
      "Validation Accuracy = 0.937\n",
      "\n",
      "EPOCH 61 ...\n",
      "Validation Accuracy = 0.942\n",
      "\n",
      "EPOCH 62 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 63 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 64 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 65 ...\n",
      "Validation Accuracy = 0.948\n",
      "\n",
      "EPOCH 66 ...\n",
      "Validation Accuracy = 0.941\n",
      "\n",
      "EPOCH 67 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 68 ...\n",
      "Validation Accuracy = 0.945\n",
      "\n",
      "EPOCH 69 ...\n",
      "Validation Accuracy = 0.938\n",
      "\n",
      "EPOCH 70 ...\n",
      "Validation Accuracy = 0.946\n",
      "\n",
      "EPOCH 71 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 72 ...\n",
      "Validation Accuracy = 0.943\n",
      "\n",
      "EPOCH 73 ...\n",
      "Validation Accuracy = 0.937\n",
      "\n",
      "EPOCH 74 ...\n",
      "Validation Accuracy = 0.929\n",
      "\n",
      "EPOCH 75 ...\n",
      "Validation Accuracy = 0.939\n",
      "\n",
      "EPOCH 76 ...\n",
      "Validation Accuracy = 0.940\n",
      "\n",
      "EPOCH 77 ...\n",
      "Validation Accuracy = 0.944\n",
      "\n",
      "EPOCH 78 ...\n",
      "Validation Accuracy = 0.946\n",
      "\n",
      "EPOCH 79 ...\n",
      "Validation Accuracy = 0.947\n",
      "\n",
      "EPOCH 80 ...\n",
      "Validation Accuracy = 0.948\n",
      "\n",
      "Model saved\n"
     ]
    }
   ],
   "source": [
    "### Train your model here.\n",
    "### Calculate and report the accuracy on the training and validation set.\n",
    "### Once a final model architecture is selected, \n",
    "### the accuracy on the test set should be calculated and reported as well.\n",
    ".\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    num_examples = len(X_train)\n",
    "    \n",
    "    print(\"Training...\")\n",
    "    print()\n",
    "    for i in range(EPOCHS):\n",
    "        X_train, y_train = shuffle(X_train, y_train)\n",
    "        for offset in range(0, num_examples, BATCH_SIZE):\n",
    "            end = offset + BATCH_SIZE\n",
    "            batch_x, batch_y = X_train[offset:end], y_train[offset:end]\n",
    "            sess.run(training_operation, feed_dict={x: batch_x, y: batch_y,keep_prob: 0.5})\n",
    "            \n",
    "        validation_accuracy = evaluate(X_valid, y_valid)\n",
    "        print(\"EPOCH {} ...\".format(i+1))\n",
    "        print(\"Validation Accuracy = {:.3f}\".format(validation_accuracy))\n",
    "        print()\n",
    "        \n",
    "    saver.save(sess, './lenet')  #saved model for future restore\n",
    "    print(\"Model saved\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy = 0.925\n"
     ]
    }
   ],
   "source": [
    "#test accuracy against the test data\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, tf.train.latest_checkpoint('.'))\n",
    "\n",
    "    test_accuracy = evaluate(X_test, y_test)\n",
    "    print(\"Test Accuracy = {:.3f}\".format(test_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 3: Test a Model on New Images\n",
    "\n",
    "10 images have been downloaded from web. Predicted the sign type and calculated the performance of the above trained model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load and Output the Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAADTCAYAAACRDeixAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXecXFX9Pv7ce6fPbG/ZlmwqKSShhd5FRIoiSFEwhCpN\nkCZVCFVEilL8iKLSqyC9S+9CgDTSs2mb7X363Ht/fzzvM1sScCdqfn7ndZ5/ZnfmlnPe99xznnc9\nhuu60NDQ0ND4fx/m/98N0NDQ0ND4z0BP6BoaGhp5Aj2ha2hoaOQJ9ISuoaGhkSfQE7qGhoZGnkBP\n6BoaGhp5Aj2ha2hoaOQJ9ISuoaGhkSfQE7qGhoZGnsCzNW92332PugAQ8fgBAGPryxFNJAAA4WAQ\nALBo9ToAQGt3HABgGF5sUzsdADB/TSsAwONuBAA4bgYA4AtavN4oLxpG1QMAPnn3PQDAww/exmMN\ntsFrWSgsqAAA9MX7AAAHHXYaAGD0mDMAAPsfGEBPXzcA4IlXPwIAnH7E9wAA8Qzv6UUSAJC2Ldx3\n/+0AgJ+eeT4AoKTCb4xUJh9cO8sFAMdxpL+AbdsY8p0k86qsXgebZvcOT/hV5zqOA9dbBgAwy3cF\nANTtsBMAYNo++wAArpp9DH544iUAgC8a2fR99psFAAhZcq63F70pPquGikIAQO+ilwEAr//tdwCA\nKac8BgDo++j/sOLj+wEAlofP5ujbVoxYJqNHl7sAEIunAQCpVAq2zf648iBV/xQncV0Xm2Y9yzku\nz/HKsbt8/BJSqRQAwGfxFQhYPPfVWQeNtJk5wzDYDlNkkkokRywTAOhsjrkAsGLVagBA7ZgShAIR\nAMD6pg4AgC/A9ygViwEAqgpKYHl4GyvikXZQLpk05dvT2wsAKC+rgmWxbaapuB6PbW1tBwAUFRWj\nrakHAFBWV8DrWjzGcoPyvwHT4j1tl3Ie/myysoCFtLRD3TNcEBqxXDwejwsApsu+/fW+3+LUk84B\nAKQd3vOhx18FAMw59lAAwLU3XQF08x277MbfAAB+c91lvF5RFQBg8bxP2aZ0BBNnzgAAxNY0AQDG\nTt8GAFCzTQMA4NMPX8SiFSsBAA/c8yAA4ILTT2E/C0YDAKKwMTpUzO88fFZPPMa55acXnQQAqKof\nBQBY82ETeuxOkclMXu+iWSOSiWboGhoaGnmCrcrQd5kxGQDw3NPPAwAWfR7H+AljAAATJ4wFAKSE\neLmywpumC8PiKr/bHlztSgvqeJ0X3wEAZJJkzSvXpbCmaQkAoK6K7MH2hQEAli/E/2N9SEPYvzD8\nTz8kmz/skOMAAI2tSdx0070AgNaWRQCAH31rP96LBAYBLvBIeUw8/fBNAICjfnA0AKCkYvyIZTKY\nSQNkLo4wC/Wp2IwjLGdz9XdcR63N6lx1fQOpRDMAIBp/DgDgP4Fs5OlfnAgA2OfO19G1+FEAwLja\nbQEALc1kCJMnlQIAPvr4c0ycOA4AsKaV7O+O310KADjjFj6Hrnaes2TJSpiOj2130iOWhUI6PVRD\nsW0HKSpEMM2hsgF4rGEYWTllNRk1llz+8a33qFH0ZQDTkKEvvzmmb0gb1LX+G3Bt518ftBlkRDus\nr6cWmrb7EOtn/0uLqwEAiTTfFW+E4yGRSSDkCww535Vxr+RriZZiWVZWdurTduT6ZdTKTCuN8ipq\n2K6RkJbx+jA5LmB4YRj8zjK9cu/UZvvkOE5WK0gm+ZDDCI1MIABcg/188pknAQAXn3chLjv/5wCA\n196dBwBY8ukHAIDDD/w+ACCI0chM7QIA/OjgQwAAe3z/RwCAWZP57loetvuRR1/EB++9CQB49clH\neE7xuQCAC847FQDwuz/dh1fffgoAcNbZZwMAJs3cCwCwYcNCAEBNYEeMGssxtWYJr330j6n1R4Jk\n7vP/SeuEVdiH5Z+sAACMmuKVns4akTw0Q9fQ0NDIE2xVhq7slsceRya8cvF8vP/BJwCAseMaAAB2\nRpiRy7XGTiUxbVI5AKCjtw0A0J2iXffQIw4GACS7adObc8IpcISh+C75KQCg/exfAABCfq50ppNC\n4NY/sz0J2vzQQfvXy6/RluhUjcXO21AL2OHY09kcox8AcOtc2tvPvvhaAEBhZCyuuIc2sb/89WcA\ngOtvu2vEMlEsKcsqbcC2FfsUOySG/g9n0/PVbwPk3ZLfbbgmH7PyI1zip2xnnnkdAGBWtA+PPPQx\nAODQ46YBAPzFvG48HQUAVNfXAx4yp/4+ajiH/5T2Qn8hv1/0Mm2VBxywL9544j25/wgFMQjJFBlk\nKkV2n0k7g1g3OzHAoAf6r3wP6rfsp8gk4RMbsglMamsBAHz41LMAgLrTOV5cv7DNpGKf/ztQDDue\nFLu0lURRAcdwbz8Zp7IbhwNk0RFfEJaPzzsl52fSwr4zlF3Az+eXSAz0WdmzU2myZiXulpYW1FbV\nAgCi/Tw+FKR2Y3pEyzEHTSvugOYJbDreXQdIJdgf5W/JBX+4+wUAwA8OI9M2LRNX33LbkHtMmkzb\n9KufvgsAuODmq9C4msw5cARZ+7wX+NsVV/0WAHDz1dRio04CLW3s58HHU6PdbY89AADTd6CfaPXC\nRdh52gEAgEyQFoHTTzoKAJCW8XnfE4/h9fep7ZspMvJ/vnUfAGDBiqUAgKMOOh4AsMO39kS5nzLc\naZvtc5LHVp3QbXGGqhevuKAQ++1H4YQL6Nxxjc4h54QCQVgBCqW/iRNJ3OKn08cBesIxNHW4V14A\niPoWl7fYkgGUUeq4FYBxPtWijDjC2q+hU2/8+BIAQLSrH2P3/wEAwF/ASb69axUA4BdX/xUAkEzw\nuiubOrHjWPZnu3OvzVkmqewELJOPbWOwo49tx5D/LRgDZhdDHcs2uBiqMjtwEEzy735RtZMh9vOJ\nsZTVResTOEom99o6Pgc7yQUsXCKqtmPhjbeo1h58GJ1Ln3zCZ1Uhmnbd+N0BAKMme+A+cj2/9OY+\no/dH+YI7GXtIXwZjYEIfUDLVJDR80jj571SVV3nkWBtYcDnNZBEfZaCudtA7nOBf2eXAnNs9Umyp\nOce0KI+0yecYKS6EVybPSIjvVn+Mz83rlUkyEIDqneNyYsoaq2SyzSQYHFDg88KVdyIt48nDW6Gr\nm4ur3yhFRy/fv/5+PviiIh5TXlEsDQV6+3jNcISLxXBTzsAzNWF6+Qx8fn+OEgFqw2zXGSeSuB1y\nzOE4/DDOKU8/Q5Ly5nt0Ij/zPMnY9Zf+EdF2mh/3OpiBENdcSfPhg398GgCQBOXYubwbZQHeY+P6\nzwAAK9omAQDiKzl5F5fPwthJ7M+nn/L8q678JQCgYTLbUlBSiPo6IQs+EsD+Tj67jR00a07fj85X\nu6UfOxzKOW3DYrYdh84YkTy0yUVDQ0MjT7BVGfqq9Qx9WttMdWfbsdUoEedDYYTLvApjNIR59vR0\nwHS5gq9sYtjixoWLAQBPVnKFTF11QfYepuIfStUzFTuR3w0DSem1+q31ijMBAJdffRgA4NGHX0Q8\nSYbS20k2Uhim02nderZv/HiuqlWjipAUh48pmkcuGM4+HThZCpVl2Zuwm4G/zWHXGa7aAkCvn3/P\nD9YAALygiaoclF91eSGWbKTaZ4uDq3EDVfgZFTRvffDGIuyxH1nrI3fTZHXkcWQ8IZNmmZoJlQCA\nZ++9AskM5ea1cmdd2Yf1TYdsxpYzXJamMNMlDVS5jTTPCZouEt6h4XkZCe3zuByPrunCcP47jtEt\n3VQm0U3WGAiIg78nDttPluy1+Nx84vB3bGWCS8FxMuoKAIBlyxg44PUqRzDHr9+0kE7I+RJKGCii\nxhYKMtTX4zHht6m1dQprb+qk0z0SZiCCtzAI01TyVX3evCxd10UgwLardz4XPP4KNeazzrgVALD7\n7mPRMIrBEzvtSVNtU4gOxicfZb87Wj/Fr35Nc8ne++4p92b71jcvAwBYBtvfl3gXvbENACgfAHj3\n7zx3m3PPAgC0TdsGjqja/v1pqmm/kmHAlT07AgDKR49BVRGv09PKfhaPolP0ussZTrz9ATR3vvHO\nOkTXfwUAqN+mKid5aIauoaGhkSfYqgxdOV0qy7naN3fHsVISILYNk/E6YrszZa3Zc8+9kBHWEE+Q\nCd4+ir8ZNpmHKaupYQIQ5qVWTNsU9mFL6JY3CVPYi3Le+Lz8re8yhiMdN/uH+M2NdI6U15CZV0fI\nSpw0tYy+LmFGth+rGskApmw7NWeZDLf3wjCyjNxxnc0ea8OFmVVEvpntua6LPpssufkG2pLDNplV\nTGyq1/dGcU4t2XvQohNs+lSGbyXilHHbxvUo9JGt7br3MQCASBVl0tfEcwqLKcfaybug+yuGpjpb\noLVsjn1/HYaHKg7G0TfQL7AhLX4GCXHs/ekF8IojT51fJdpal9jZPbYB+79D0LeYoackPNdwVVJU\nBoY4z0wZ5+EIn3UsynfG9bowJZFo2WJqxkGf8mXx+fuDbE8kHMKSxY0AgJ4u+o4mzZjC/6Mc91Vl\nARjiKK+qJCMPFvN9bmlbCwAoKJoIr5eajnK8Km1YYXhiGDAQXpoLZkwky12xkkk6lmHi8eeZ4Dau\ndgcAwJRx9O0cP4eOylceXw63oEbuqTRbXm/lajrzL2pZK/2Oo7aU1oMpy5hYVFxB5v9ORAItTB8S\nDmWZlH6Fr7wRAPDJZQyU8PiPwJhp2wEAPlhJTWHmNt8CAMQifK4vPXQvAKCgeDJQLhaK1tzeH83Q\nNTQ0NPIEW5WhK17SUEO7UOP6JkQijLiYt4TeXEvsVK4smSXhIHqESdwWoOfcmyFTdLLpxbyy59o7\nEXHInC05P5AhE4t5uHL6HRdt1zBFXwVgGMLwvQZXw5XxPqQlxCssrHRdK0MmYxmy0boiiYhJ9KCm\nhklR2AKbqy0py4NZqe0MTZcezuhc14WtwjolQcbE0ASeARsqEChgQtBGMMTts72Y3j517kUAgIcP\nPBgnfkW7ukdCvMpCbE+il/3d2NyB4hJGMYybwpBOlV7e28tP1yZz+/TxC7PJIu4W2EWHY3NRIcO/\n29wxbfswwsAwKRtb0uJdrwGf2I9V4szGMxnlEPj9LQCAwz96AU/tfui/3fbNYYsZuoSzuhLOGc+k\nYBviv/EK05QEKcMme+zriqGzt1luzGcywJrlvZFwwUAggKD4spa00l/lbyMz97vUvjxmEGUFoimI\ngbw/Q39LSTGv09mxHs3NtPdPE601neY7nMmw7a60AZYJx1HJM7njF79klEpBSN4Hx8SsXffljxJy\nuXH95wCAyy5hYlki7WLvnfjOBuX9Pu647wAAin7FkMd+8Tt4I0EkEmzzhxX0OXki4qeSRCi/18r6\n7EyLx0Rd9j84l0z9g6suwZhO+v52PoC+qHnvvisioMZUXs1kpFH1JpqE8c//xxtsH741Inlohq6h\noaGRJ9iqDP2gfZm+GpM07vmLlqC2mmw95KcNvamZP6YkueS9LxbhimuvBAD45p4HYICZ+2U9Cl/P\nVdBxTRjCWBXDj3tVgSuykZRloPxqesS7r6UnWhGmlCSeBK/9OebezDjSv/6FcckXXsMCXHf+im1I\ny3384dLsqtjSzIJeRVXBEctkeOo/MLgA1UBUy2Dwf/Xlv7Y7NnXSu95v0uZZOYmaTttxZOjejiPR\nNJ8su1geTkLqi2UM/l9VW4f+KONnI0VkVH4f2WAyQBvqazd9GwAQS6QQiUQ22/aRYCQMdnhUz+Dv\nrrn3TwCA11VxLpvPKnEik4f83sBA4LlAFTyzJDkmVlSyxUz6vwVlYzbkM+0a6I8p/4DY1X20nSdF\newIMtPfxuYUkqNyTFG2umMxwbZTa2f2PPINx++0PACg7kKUuAhJk3raaLH/FyqUYJTJyu8nMbYms\nUdp1LJ7Bkcf8BADQH+uWNvOWpipnofqUseF6VGRR7olFKnKnT7r7yON/x+yjmbh4021k291RXvfq\ny5l/cuvV9+G8q+YAAE6a/V0AQPCKqwEA69evBwCUi508Fu1Hb5SaZ2kptVcVueOVgoKObQOSY5PK\nqOQsKXlg8/0JXXAFlt7EPJWP3/oQADChmGU2DjidJQTefJ75MBn/Pli6kvesrC3LSRpbdUJ/4tFn\nAACSuIajjzkCmQwf8Etv0RlRXk5BpsSpc/75ZyFwB5NU1MtricOl8LIbAABJqWNiAjA8QxNOlPBT\novI5GReQBIbglay0lrmaSQkqIzNketDfQ5XzkvMvBwBceS5D9BZ82QgAcINU1as8EUzYWR5eqiBn\nmWwStug4GPAjmps9Zuh3atIfrmwNOFubbmdoV/ltHPUbJ3CwGNLcoqY+PCBJPL8PciJOyku+YsFy\nAMCEyZVobeVC0LiGL+n0maw61/gJa2a0d9EsFfD6EZd6LNbwmTMHbM6MMnwi35xsXp9EE1NKZONR\nnwafk2VZ2fM9Hr4CGZVlKlm0Rto7UOXyP+wcHahkmBtau1hlNBTiM0o5Luw+JQdO1pZHEoK8A5Oj\nR0hNT0aqKpby/HiM79jLr1P13/XcsxATk4tXTIEbCigEX5GY4iaMQ9cShrimezmhjw7QOZ6Sapz+\ngAvLKxmmUsvFtaU9znAnn5nNjO7q4rgqiIy8lovCk0/REXrSSafjqIMPBwCUeGgmev5lJvs88CfW\nZDFtP87/2RwAQOJMOi17JEmxpobO0oyEuHoNF+PHcjw1b+T4TsWkDxb7FioIwZOWOUDMWKYEd4SK\naIJJpFKo/g2T2YrizFLuuJzzz7tP3AMAmNCwMwBgzdoKpKOUrVGWmyy0yUVDQ0MjT7BVGfp3DmS9\ng1iMDPGtf7yNlg6yjv33p6Pg06+YYh8Slb30rmvRLw4UU1hV8YVUjxLDajcPZnTq7+uuvQIAkFIJ\nEy5QXEI15rIL6AjrlvBF15aU83QGveOY3nvXr2mWifYL45d6JtNnsuJdIpaBJVTOLc497Mpxh9Zg\nMVwTMFSVwaG1SbLnOANJGAYGQhl5HX56JbQt6stgno9VLpt+fCzkwgCAuHx2X30znr+BCRFnryb7\nmllN1hUOsb8FlovwxIm8Zh/vsbaRYVylITKr8TNZFyPTvRHNa/7Btm4hGx3cl5H+9sNjfwgAaJHH\noMbL2gtpWqpWqf+mscn56tiOU1lLu/hPt8O7Pevwp75YsIU92DxyCcscjJjU4LcsqvppAzBVOr9o\nqY4hTlEZF8l4P7wGTSyFYhYoDpOF/+FJam7fOediAEAiXAATPKZbNJWAOFelqCO8oWIEdmE9fUsc\n700LKJ8Sn6rmaCCtEm2C4vRX5qJsb0T7dAYc+C3NZMANY2pGLJPXnruTfTiU1RJ32Wt/bDuFCTod\nUppgvz1Y5yWTIOt94a0/oFLCklVAhQpjTaoqpfI+JeIp9EmtKEdkaohT2hNm31LRGEyp4aKS2dJS\nt8ptYnmMiGUhKlYHIy0lA66nheHZfenwXLuBx95x2z3oracpumf5FyOWBe+voaGhoZEX2KoMvb+b\nK3CkgCvUtKnjMdPD1XT5mjUAgFapgve7G5kUYt92JVxx5hiXidNAHJK3/voafi/L/sWXXZstThUX\nG3BaCmlfOfdXAICbfn09OrsYipUUdtx3A21ZkUvp8DT8foRmsziXIzYxVQ+6WByeTbJrS0HIi04J\n7Zr3JcOjZk88csQyUSx8cIji5uzqQ2EO1E8fZCunLIQhGGRa8XQ5itK0JS59h+GGlpfruFf6n3no\nIQTsqwAA9VHaCxt7+Rz6Y2SArfY4WF1kOKkE5T+mpgEAsGjeW7z+Ospkl933RlMrEzN233mPEcvi\n30XPGbSH2sqPIrXYS5V2ZpGF/eamm3HJxb8Ycq4Kswyl7Oz/Bz38RwDAs1N3+4+2c0uLcwWEBWYk\noQ4uYIqdOGwKQxSNqDfBdy0UcBEV23ZhKUNtP3j/SwDA5H0YvtpdShbupF0YwvEC4tsyLMrQp7if\nYyEp968SDWb+aobjFcakXYaLqOw4FvTznuo9ymSztSjvRCYBjypCl8ndCX3YMRfy+qLVfvzBW/jw\nvdcBAL+8iKn5ZZVM7/9iGQMcxtx6B9LSB0d8PUFTVSeVKqNSbbOgMAxPiDK2pcqoLyhaq/QhlkrC\nFKd0Su2XIDNrpI6JiT1t7Qj4xeEuxwalMNxXq6jpXnox38EX3v47XnmSdv/rrvq/nOShGbqGhoZG\nnmCrMvTXXqNdFZLI861v7QFDSuPG41z9zARXzroLGWK21nFh+sk+fC7Zwg2/ZkihLYkDHol6ufDC\ns3HzrbSp3XXXHQCAM8/m7iWqoNGll87FlVfSpqqIUrmUa02p+uOug7jUjPaLDfnt1+khnzCN6cTF\nfkYMxFCO8hK2b9Ye++Usk+HJQw5cuMMY+ab24n9tWzYdahKLz/wl+g0yzLDL/QnTEtmhoj9gZlAt\nNvcf9ZMtvFhKZlG5M9OVO3sS2Wc0dgxt6TfNJaMo9zMs8rAf87l89f7vsN9R/LvQmPdN3f+PYdK0\nibB9qhAbx1CtlABeK8z8ppsZrnreOeciXEj5qNrg2VLFUj7WBw8MGXf/KzDFDu33SoKLJ4AUHwkK\nwsKEJYQwFuO70h/twroNjQCAyhRt0/PX8/9p3yNDj6sdgqyBBBlX7O4eiU4xZMy4APwZvm8bApR3\n5SyGI6c/5N4GZiaDvigjogqFCauoG0NYeDKhkuecrGE9FBp5uK+C2t1K4b7f3YN3PmGN9LjDcNpF\nixldV3QDfW9dbX0oK2OY9Po18wEAVWU8tref73X9xAkAgOY162DK/OKRSDJPUJIKr6d/LZNMo/98\n+upqxD+39ufcR6FJ3qvqUTUIFFLL959HraI/TdnsuzsTiu67//cAgF+ccw12mr43j7HacpKHZuga\nGhoaeYKtytAbJnHVq63iavjuex9hl53pMe+XyJeiCq5+G2THFRgO2sQL//tLFbPmOnTJVYxPP+xQ\nMo09dt81W9hKxWGHJWbXVgk8cGFYZvZvYHBhLJW4AahNg2xJcKqU4lWOlBDIeBgvH8zYWNbEVXT8\n+OqcZaLuM0DCjWy7FFyx1Q22vSpbnym/qQ0u1DEZj5RKnb4vvC++zXv5VeEyajZq31bLDWDJqSfz\nuz+Sxbato8e9pJalSAPeNArK6Hlft4H28WOP6R1y78IKsp75r8WxZBHtmDvueszIhfFvoOEP9yIt\npRsUC2y+iFoCJLU9LDbMW353G847j5qbX3ISfnkFtY1rr2HyR0NvL1YVqU1XeBkjdxNvFoOf3ZbG\nodtpvq4V1WMAAOmUASNIJl0kJWgzwgjboxy3ReEiuF4eXz2az9JZxhKx6ULxG0jyS8L0wbYlacgR\nRm4pBqz2HzVhq78ld6Gyntdd9z4LZBUB6Oln5FOFlJa2bUnNz6i9guX6jpnd07RQGGxuMmH/veC5\nPzlvDi67aC4AoL6M7+M7q5nIE1/OnckK25qx4VrGrVeXCNs+nf6X1G9+TVlU8H1PtjVD2futC8ms\nA3cxQsa4lAmP7ZecD++1HGudkqA0bjxLC7SfwV2O/P4qrP0FS450SOJTw530CZ54GSOr4qv5zvb0\npzFjFneF+sM9z+QkD83QNTQ0NPIEW5WhQwoHeWUPw113noVIAdMVS4vI0M+8lKsg5jIu2u+aGCe2\nzFCQq/Cll3MVVEXzX3yRpVr32XvPLLdNy8ptjyDm13bIUDJQ8e5Wlo61i7d+jZS8nF7AFX3NWq72\n3XELVWVSLCyzZfHFm2Lz6+zm4pez0S7G0P9j0pdE2ou2iWSjak/KkPgFXnvlJQDAfvseiPrX3gEA\nRA2ycG8l7c/RLhZNy8R8KKxltqDaG3ZDGxnaxDFk5pDd5Xfe72yYBexDQXHJSDu9RSit5qYaKU8m\n++x9YktP+Gj7DEiJgsIiFhdzbODRh8jQjjuebOnquRxTamOGeWddiKJHGeXyvVWMBX5u7HYjbtc3\nFhTbwiiXMdJXw5ZoC38GQS9fYUuKjRku7dBBiUdPOw6sAN+fvi5GIY3ZkSnnaSl655ECdj4rDQOy\nn6vYpk21h2t2owoHthT5stLym7ShLyARYX0uEnG+f5ZknEIYvyrBYYtWAMOEI3/7fbnzS5/IsqyY\n7W3t9eKPv+dz6+lj9Nk5ZzH/wq7g2F71q+tRczf380x7WBahaC5LexTfwazNfimuNeo3t2Pj8ScA\nAArHUtNp2sjcmTFiWw9feQPazmfpbTPBvvT2UtZhkf2CN97B9rW0TFTOPQkAEM+wv7MPYUTdCScw\nj2L/Q4/DUQcewWOUg+GRkUW7bNUJvaaML3drK0PiUk4MDVJT2JVqgeUz6bjbqZiT5LMrVuHYAxh2\n9PGnVJ3SKU7+HovHXHMDVZ+fnXMBDpeNo6vErKMqKR70XdaoeObpF1FRznYEw1S3Mz1Sy3gD61UE\n6+qxezGTLxZ9xnvW1PF6ts2SAPEEH/ikiQ0QfxFamtcBAOrHV4xYJpvba1FthOxmzTFfXxM5m56u\naknLSQt3Yn2IjGWgO0XTiCEmlpjU+RjYeNlBh2yiXWfztzMy/HykjKGOrenO7PElUnVxw3qaZYJF\nlGNHDwdzcUMYvf2cWJa/wQp31SfNHoE0csfBj3JibjIAW/oXPZPhp8GQODWV71eGezDky06ujz36\nOA9R8pea6XPmzIFPFoK0hJb6VM2fb5iPv6kKpDK1bGkpgUJxxiXSqsKiD15ZgLzi+LVkMJoyHDwG\nEJLJPi3yiUgS0qo/PgwAiPlUGQQXhpRHUOUnHFNd15JjPPBLKRhHdqNaJ7XYq8pIzjzxOKJS/yQt\nJhdLVgTbHkpKfD4PbKnP726BwWDv6QydPPJUPvOyUUWYujMn3g/f4zzz0qNM4Jk0igtiKByAx8ex\n2tMmNZkuolOzX5yits2x3d3Vi7JSvs+WzbDdkI9ju7+b80aRP4zQaJp3+mVv19WraZac9XvWFfJ8\n9BHWFslm3Is5d4zflSbom39N03HlOO4bunrFPNzxABcWpy23SpTa5KKhoaGRJ9iqDH3NCjpjAmEy\n65KKQngDXIHiaTLC7Y9hDer35jNJpzhUgBdfoTkAYk6YMIFhc3Pm/BgAkJbwH9f0ICHVAhcu4O4s\n8RhX3JeYpolbAAAgAElEQVRfYsjkhPF1WN3I1TPeyxX2O4ewJMEztXXS0gQWtJBhXLQDtYN+qQ9d\nXCisr16OjGWyTO7P9/4dALDTHjuMWCbpYU5RA95sivHAjkVDq9BZg5xzyglsiGrWD6lnfRLDpmrX\nLcA6KZCUEWanipsVFJJR+awMov2U4Zc7cX/Dps+Yzv3xRzQ1Td1jGvw2mcXSJSzYVSKFnFYt5XN9\n9SWGdh5/yllYuWoRAGCtd8qIZZELvMIYm0OUvZNxYAhj9Ppp+skyYjnHluSWo489Bk889viQ61ni\nKFfJLSYMGKaYHYJSSW8EzLphHDXMxlWqXMCmVSG3NLHIcNiOsGStpF0Xfs9A/3mQUGvZIciwLFje\noc7woCTKhFv4biRkz9lg0g/HFe1NOd2l2JfaGcu0zawmlJbvEpLyX/pdhtolG/sHilvJvZOy81W2\nqJqYdjyeAExVO38Ldiw65iyy240LaAJtj43H3x+giXGnw08BANTWkLG3Shp+2W/vwcoTWCpgx+34\nvNbNZiXG4kKO6coEZfPWurUYLTut9d7JgIHAOAZEFMhr2dnViZ4WagOqEmXUK+UMjqdTdNxJZyAl\nhcm8jdTkXcXw69i+2Udx96W77vwV4lFeb9FShlWeiKNGJA/N0DU0NDTyBFuVoYfKGXQfEmfV6tWN\nCMvOP4tWMPTvxTBZYEhWypbHX0K9sCdLzlu7jqvXm++QuYf8ZJrXzr0Mv72dzo1992GwvsdLVlNW\nTrvv0qWN6OujwyIlzpi/vs7SvTVTWA42WV2I9LVMUFp/DXdc6WwhU23voUNK7YvaH21DMkVnrfEN\nCT8jBcvnDi0HMByDw+fUMR4JVyu0yZaNNPv9+vY/wPpGpmZPnsIiXT5h6Kbc5/XXX8e3v0vNyNdD\n+dsW+1k4no7QpvlL4KvjcKk26GxqBtnIg5eRWRx/A+3ZSxavQryLbKja/ExaOmvEMvgmuNL5n/zl\nXgDAemHhlt9E+9lkWYXipFOlcZU8LWVXNi38WJyhDz/4kFxZ2ZGl4JvlQffRvF7kMTqkfjb/UwDA\nHTN2+tr2zf4hSwpcf8txcm9pt+tm2amxpUZ0c+i4sFyw2hwGduhyVPq8K+n8Thp+2UM0EuF7srSF\nbLZJ6tqHxT9pe4zsbkZJPlp4lDYhLBqWCze7PygPMqU/8RjHYCaTQjzGxCJDQhst8WXZEhrs8Q7s\n86rk4vPnzi9XLyODven3DDu949fPYcouDEFc20iH/vLlLDg3uYJqdTTWjXF/poY27+SjAQATJ73K\nCx7CMNtPruMzt8bVoUBCnyFh1847X0rb2V9PKImAONxLRJtDJW3qMXGOlhR4EO3nO+GvKRRZcHye\ndgL9S0oOXakQrjyHPoEv5jXnJA/N0DU0NDTyBFuVocckpKpIvMY7jx+PYkkmqKtg+ri3UEIIVaXT\nNS3ZZccetoP8m//4AABw6ilMiqmrq86GSxUXc8X0yG4o7W1cHY866nDU17EAlVfYe/mvmLiUkqQH\nO+2FT5jK+AYmCFTWMjSvUlKGvZZiGg3obqMtLHjoYbmKZEikCcDM66/bS3TgnE2/S8m+hO82MMon\n5pVIoCID2223PQAgINEOSpFQtnSf35/dpT3jkJlXSYH9iwJkVrcU+7GKm7ngxp8x7OqXtzFc9NS7\nVwAA0l08YOaMGiz/rBEAsHTFin8hgc1B8YzN2FTFRtk4bTyPkIigQMiPItHgfL6hNnQV6aQYuxXw\nwSNRF7Nnkx098cQTAAaPMSOrDfjlHk01X5/4ou5VX86EkEnTWW5h8ecfSBsGSvaGCnLbhSZ7D4lG\nUYl1qVQKTlazUIk7vEc6mwjkwJIoqfIwGXqmm8xyl58yHC8jPhbYxsC1hUkrX8XgQedJqUJbQ7XD\n9i/o9woF/NkCYorY29lnKrb4bPlnX3ZnruHv90jQ2U62/Pwr1J4+/+B9VEWkzHAB/Tc9tXwmcSmf\na//yPFi33A0AGPMANbS2sxl2GD6AJaAneWX3l5O/i7SMg411DPdMriO7H7WeloLma6/A+D+z+F+X\nFKVLir+idiwTlLpa2uGTonGuJIitXEI/y9sfsjzGvrvyPQ2Zfvz2vqcAAB++8xoAYNr0Y0ckD83Q\nNTQ0NPIEW5Wh77cbEzNaOulJ/+f8RSiIMGpklMSN2ykG7RvCEFKFIbh9/Zu9ninMYrvtVMLH4GgQ\nrqqn/5Ts/ZZbuL/gbrvPwnk/vwAAssVyHNnR3JAd1IE0LryM5XtTsulFqdgfe3poGxw9mu01Uy5e\n/4gRNduNzd1OnMmGjyvvv5ONcnE3EyEBMNIgS7OVTTIhBfpPpw+hLk4ZN2Vc3PArlhnefgfKyS9R\nRvvsyWJif3vy0WzSkSmJIIsPpg3Yeo+ROx2fN2FKPRnP/z1Dm2RMtvV79VmynF134Z6iGTODDoly\nefMfDwIAjrxw5GVAd9mH+1F+8va9bIPPA0cezZw/MGlkrWheyha79OfnqMCjLFtVMlXax1k/Y4q1\n3+MdKJ0gDFT1f/bsOdlzMhKVoIozeYWpZaNnXDfrz5g6hRu01EqS1b03M474zVfI1DqTC3HznRyL\nl5z8/IhlMRi+wLAYd48FS+1vKabbZL9s/WapSKn0QHKd5Hy0S0RZFagxp0L0V6WSGRhKqxHbd8Yd\nWnbCdV04QWHmkrIfj5P5Lv6CvppdSmrgSOnijNJyJCcgKRp0QEoutHd3o6hIbWGXzlEiwMQpuwAA\n5r1PTaimthCZEPswzcNEoqWSJBUWDWVZWwdqelmu+8OTmK8x9bHnAACOBPB39vM9745lMEq26psw\nnlrh+mI+486b5gIALNOP1oW0AEzdn1FxbZLX8eU/mJQ2ZrvJ8MvWehvW8t7b7sOooC+eehIAcM/d\n9DetXPEItq1gWfGl/UtzksdWndAfe+BRAMDOe1AdPXCfnRAQh+a786g6WSmVrcmHMvXHP0Dsbk4K\n11/HegmXXzEXwMCLmojTvHD+eXOzbsniEoYWjRvLh3n7724GABQUFqMwSHNMR53UB5dMUUPtTTrv\nUxTsyOytUCHv0dbMBzxtBkOMetr4UF58MQE7wnuNHZf7XoibJhEBAwuTMscMtbFkBod3qYVAEiXW\nFXGS/Xgyd2kZbVn49rc50aqkiZPFRBWPS6agx5M1B6h6KOWSGGG6NA9YX/bDr2YPSWL55+t0JB3y\nfTqSOrpastcbM42Tf/k7PxihJAawbh0nhoBk5SZSCZhSU6Rp5+nSBjlYNrOuS7rwBzhBqTohw/cd\ndWSCvu+++3DCCScOuWe2SqVM8IZlwCd/f3Uas5en33sL29VAR3m8cRkKxTn+nb3mAABqxkrNazFr\n7P99/p9JzMLC5Ry/06dV5SYQgdpVR8FyHBiOhP/JBudiXYJKR3HhZmv2pJOcyKdU8v5JmZC7JQ7W\n9Pmyk79HxpjKnFVhsTDd7H6ZahxGZG/NPtnByhcZhZSEO9riyPVJGJ8bk/Dafr6zMGxEpTJjMJB7\ntcUKSXxLygS8sSOOeDfv+c9GJvWkJSRXjffRf3oQzVfS6TjtYVZmdGQucWT8e25gOGRBRzcCpVzw\n1HiqvY21XGJdzDItLa3BsuWcD5Yuo+m49AomM02Wypg9iSi84jBVe6g2nsJ9ikv2+w5lk6bJqq+3\nE8tW0tTy1osM2PjlxReOSB7a5KKhoaGRJ9iqDP2II+k0/GoZw6Z6WtuxWupvt2zgymaUSIKHsNOu\nSWOheIliXjdcP1d9AwCwhT24xsDekHNOZGJNRlKPfeIQtB0HQUkw8Z1FlqaYRlrVZX7kJUz/MR2l\nvZ1MKCqvZNpwXOpMr27kORO3L8H4OpowPP5OadfIqy66m2HoA05RddRQVdsdUvaPMnjfZPv8Ugd9\njISNpXzAfvvtCwB4+hnugrJCHJWWXPfWW2/N3lPtTOMXU05JN8Omfj7OxJ1ffgUAWNtBpufto3ns\nkw9oQth2KssrzPtkBaw0k798Nbnv9tPdTe3AMNiXbx3wM0w4hwlPayVpSO1uX9/Lqn4bIpEBlr1J\nNcOh/590yul48MH7ASDL1E87/Qy5rpgcHDcr9lG9NPmpHZ6OXfoKACB6wp8RF9OEqujYtoHjrW6s\naAtCld2MB9PHM4ll4RKyuUOzRqKRwTKHJuC4ppmtxKkS1NRO9La8K3A9sMTZ3yd1/3eZQi3n/g/e\nBwCE9uEzchGGJbXOs7WBhPF7pBZL2krDlR2QVEmEItmx68DdmFDnJNNZiXvlQgllNjUC0nYxxRgm\nHLmOnVQmz5HDMGmuSQY5DqoKa7Cyg5piSQmDHwp9vH5StJh0xkDkNrL3aB81BZVY5hOtJZEUph4q\nRHcftd9CqWgZ85GxW9U0vaT9QRSP5/llpdRo25s4n/lkv4dIRQS9C7hfcvXbL/J64+gEbd/ARL3p\n2zLBccUKB1PH03SzdkpTTvLQDF1DQ0MjT7BVGXoyyZVum20mAQDsVBLtHVzJMuK48MtqHZWU4YwL\nBIUtXHYlK+LdejNDhJ54nKE9H330TwCAaRqDUugJrzDzLIGxLCSvZl31zLBKhVUxrvIvv/0++vrJ\nFpauZCje5Mmm9IF2sKRN+9nSz5dhdBkTTVLikBo1euQyUbXPnUF2ceUMdbOtHrruuoaZLXIUVNrJ\n/zHsDlKvPWqQEVm2hW6xIbqWNeQ6qrrkU888jxdfYuXFww6jFpUSoSzZk6y7auGX+PSLPwAAjjyW\nlTCL/XQCv/3GGwCABYsYhjVtu/GY/ymZxaE/zH0Xp3PPpGPX7yUTaigtwFPbkIlZEvrliLF46bmX\nUw7BYLYEg4KyeSrbuUdYWDyezB4znM0/cC+r8M2ecyIM0YR8xrACSaJVjSrYGWYhx8HYKjKzR5+m\ndnHa8WSHS+gawpfL1yLWz2Pi1pa9dqrvrqu0CAdJU8WgyqeKTFWFHZ2BPWpdCSVU4Z1lCdkvdD2T\n+voqTHglSCEh+2eqZKa4UHUzY8CQd6pY0uMfuol78l57MrWcpfPnw87um8m+xqRuQlwqdXolBDPj\nugMO+WElLkaC2gq+bCcdynDdvzz3MmZW0HkZ7WAi0IrlfN7e6xkc0H3xlYiLry4j71ayX7ROKSBo\nye5JdgowpCpb1K8qWvJTbV3rZmx4JZmteQO1r1LZASnWTRmFU8DoKh6zbi2dyDMOYEXFxYu416kh\nlU77on3o3MA277X/QTnJQzN0DQ0NjTzBVmXozz1D29FuuzLUqKAohJiEBFWO4urUeDHZc/mN9EJ7\nPF7gmMMBAN6HGFp0wfmXDLnu5gofucNCzFSYWnt1DQxJXR7gcxRD71VkGvFnj8SnnzKFfp+9FMPk\nStvTy5WzZR1X2YMP3BnJNH+rrMx9x6LN1TgfyTmueONjJm2Aa1KMsHlv7+8CAMpUZIzhZgt47bUX\nyyE89hhT9H/0IxYoCgSD+MEPGI3yhrBtFRlTkaYtvjgDrF0rO7lLcajFS8lGd9qWNll/FZ+hZRmo\nKqUds6c1kHP/Al76JHbbiezrob55gJchlynptyWaCMRWbJqmIs6DIGnxltrRnd8+/Mj92e+GM/ST\nTz0t+7fy2ThenjhGdq9aJ9SsrTmOcJJMbLWY0pcsobwuvY7haqUBymTOsZOw6CtqgC29ue0Tme2N\n7JU6UILWhU9GsatS7DPsV1CSq+Lx2ECNfCmb3NNDX88PDqD2dccjHA9jDjkQMbHBe2XHMEMy/AwJ\n5/O6SVgZ9vGjvzH67Ef70fZrqd2zbCdbhljtUapKEyhpJyXixB/xQlWH3pJ34cmX/wIAOOs0FuJy\nEh2I9jCRqDXFd7hhW9r2434+i2WnnYjie3ieLyghyxJmmK2cIM4P07Kz9n6vR8I8JZEqLkUBk7aN\noJQqcQJ8HnHpSkgSEXvTSbRezlDofQ9h+KydEV+RMitkWCagvzeGSduxTMe8BS/mJA/N0DU0NDTy\nBFuVoe+5F5n54oWMctlpl+lISUxmy3pGk4xzyDh7JKEh49hI7sroAOdh2sxVKnbG2tTmpjzmXlmr\nMnJsegZZpOdHA+n5is23XsANMj5+jPtghgs82HEnxhoXFAkrSsvqLLugH3zovgCADz76B/bem0kl\njStpMK2ozT2yY3C6/6Yp/8NLAbgwpfjSBoesIxRibP3YFYyc6JfdXyzLyrJRlVo9Z84cAANx/JlM\nJvvbwQfTFqlkI2Ht8MXW44XLqT3dGCJD22FGAwDgovOYNn3KqSwJ0DBlO6xexgiKQNlMaXPDSEWB\n8iLKunoc+9BRvH02SUiFwjcdzj4UhSQ5DGaW6ajonSJJHLvjzjuH9D+dtnGKxOIrVmhJZo6ytz/w\nwH047TQVKcUx8NmJ1BrLH+H1yh/YDV8eRgZlbeSrVColiR3ZQ7OgglrNvY99jNH1RXI9/4hlMRhN\njYw4CofZr0AggoRsKhJP8pn4xOEUDBbI971YvZpRFNF+tqWpkunotXUsa3HKYdwp576n/w7/dPq3\nQlNpf1b28oBoMiUpAy//mRFC29fQJzBB9iptkx2RHK8Pptjce7qpyXoCLPJmK7+G9CmdcZEUzcdr\nDvNVjABlPt47Ui9zgeOF42U/J9azL+tXMb4bJueYQ446Ai9exmdZfTP9QsmsFidx+KqYWCgAj2j7\nqlywY6tSwKIdegw4Ml85Iie/+Dk8EofuueIiTNueBfBu/AUjqyxJ/pp9GstP+HyiXRVUYsFKaheV\nZZNzksdWndDLJAlj+kzZtDccHEhcEOfLuZcwoP9mGfOplI1+UwT4O+4qEryATtGQqgWerdlhZItH\nqAy17utpnlEqoMdmjWgAKJXZ4bEHmA25tpEPvKqqDOVFPL+thQOypo7JQ2Wi6sfjVLumTJ6FlAzI\nQKQmZ5k42cp7MtBtJ+vYVeaAjDN0QjfgwiOqtXknQxGj4rhrDjBcsEScUJbrZGtlW5JgkpIEE6+E\novlNE4badknVsVbKm3ws3PUojFn4MQDgl8eyKuW9U+gM/vZ5dJL+85/c3WnMNtvj+8dzMnztH5/m\nLJMCS7YafPwR9vuko+BIYko8yH6Pe4lmgsTjNMN1/u1ZhCwuBIUS7jh3LseSWgxUQsecOXOydV2y\nkAQdNZZOOeU03HPPH7N/AwNhepYssDFfCoU+2UhazA2RQt67rpYOx1iSKv26ljYE2jg5NlTnboZi\n+9m2Xgmj7OjogqVMIvJsXdlhqDBCVT8W7cb4Bo6JgKoEKGOlqJhjOhJmm8+ffTLmLWTW8yt3/w0A\n0BqneUht6VgZKcJZh7HeiVccn0jJDkhS+dC2M9kohI5OLjQR2QHMAU2Dykpq23Z2Es3IM8gFFZNI\nYBxZdMsiE/HqFyR+swpoaknEKAu7j6Sn3exBlYS/lnrZhza1RZ4EY9gpsQNl0kjIO+GRUFnLr6pM\n8pxMJpWtER+RmvWWFEvvOZPEYd99TkNRGeW9RkKhj/gOTVWmmLMWfMX3x4EN16AZ95N53GPgxz85\nYETy0CYXDQ0NjTzBVmXo/5zP+h5+2Zy2qnZ3JDJq9eOqVyq1m73nMbHHufEqGLZSp/iR/g1Zd58w\nFlWz2WuYiMkmrR6pFQFRoS1LrcAGYhkes9O9XP0abqcDMChOsFg8CStMdbJVqjS+9A8mz3zv4H0B\nAOFCtreoyEI0RhZfWJR7Fb3haf1cY1Wq+vCjVZyUgYQwiw9LaXJJ3MzSBqUG+20GVGW+gVBOr7JX\niKPKlvIKSTsNUzmvLLUnpVTAkyTymvaNcEXF3OlUpiNPK6Fqu2o1WdjylQzZOggGlixvBACUlIRH\nIIWhaFxEmXt/yVrVSZiwRePySvZXRkLJwrMZ+lU0+wew3KFaxbXSh+B6JkR9/rNrAQAh04U/qyKz\n3+eeS7PRjjvSRJTJODj1tJ8OaZfabBpZR6qLQDHZbW01mXrnBjLFzxcwHLfAT1YYcSPYfirD6fr6\nunOSh4JPqe9eFUIbg6XMSEJuo/1SQyXJ5+ZHCeISEpwW76PHy4OjvX3SDf7vD2ew7TZsY4mUGUgb\nfFcLpZZRf7QfPnESpiUJaf3aRgCANAu+UBiJJNsRTci15V1V8vYqxdR1s6YwZdrKBY60paGU7Wts\n/QKzZnFvzt5WtquyghpKuoRyT8ZNjK/jDkDPzeauZ2X3yd6ycRlnyilquAhYfNe6L+M7VvEb1l63\nDbbX57dgSqBFUGoMxS5gCOeBh1JTXTRvHiYVMDnurlup/Zx9Pk1ds/Zg+G8owKCKivp+rFnGEODi\ngtzKRGiGrqGhoZEn2KoMfcp4OjB6urhSRsJFg5JniOoarkh/f4KMeK/TT0DoF9w5RiXhWOIQVFub\nKwYaczKATzEvsSGLc8OVVPYIPIifROfeHxZyF6K+Lq7Ajzz2DADgrDOOg2XQOTt9W9rFx09gar0p\nakJa8i7WrV2DiePpQLJV/FUOcB2Vzi12ctPNOvfUPo4DiUZyjmuhTeKrCuXbRXexGJZiFq4KEzPM\nrCcxrSo7qutIMoptWtnEpmwYn3zEbbG7+21UGUz1/9muTPWe18KKcCu6KccLLr4UAJBM2ygvp71w\n8rjc7cUPfckEsskGbdi2x4Ej2oSYiOFIin62UJlpICN+EpUinxI5xhpoZx33LB1gHrjZYzxSkO0N\n0dquPGoOACBiOjCExauiZEo2G4/gjjijnrodS8J0hE8MMCR0zEyOk/Q82kBH11IO7S19aG2n07DA\nt2U8yhG/SFRCfW0noV4BuGK7TfQLGxcNzuePwHWjcgzZtrh8YEsKv0TLocgB/IW0cff388uE7AUa\nEO3Ab0Sye2PGpNib0lhKiqmhJjNJ9HTTzt/VzvMLioWpy3AwpVaBa9hwxEeWzuQul76NDI99S8pQ\nVNRXItrH5KCNa9nOirFsy6QKhjMmvZ1ImzxmvwO4w9R7Jx8PAIhLNUi170EaNgzxOfhFZU5fRM0t\nWCBFu2CjvZNaeq/sP3r/EywP8ezLLwMALjvneXhk79V161l8bvYPWePcJzsiLV1Cf1Pd5MnwS7mG\naI4y0QxdQ0NDI0+wVRl6STFXxfmLadOcuG0KrnjPVVGtWDeZQGkD2dHjc2/GMddwFS286EwAA7Zf\nDCrKBQCejJEtH+qRLw2xyXdfyqD+F558EmXLaOudN48rZaWUE/3ZOQy78xoG2mVPTEts+x4J36oo\nIYPp7OWKPHHi+GzSRdiXe/lcW5m1B5fPVSVxFZVWoV5Z27eF8mv+CgBY+QAjVrY3GZqVlJCqpNj9\nMsgAUu/dNYbdTK6bgZPdD9WQNd4RbcOQ6I2okQBAO2WikKzIWEuZVJeWyvVpT+5u70QqyXa0ttCW\nXF+07YhlcsfFTPp4QJ5lr+UgoCJ0pAsq4kQVj3JdM1v61Sv9zNYBz+40r6KijKwsk/LdGrGPj/37\nXQAAyzXgl8EUUOXEk2TG7x7HevoVJrDH3azd3ncmx1SRDIHxY6jZbZQoqUzGwPomlpHYedqkEcti\nMKJdKh1fwvy8gezOOAMRTHzufpFBsLAIyRRt5Wk5VjFFV6JKBhKPMvDLvp7JFO+lao+lM3xPDduA\nKzsWrV3TIu0Rf5Uwf8M1UVHUAADwZaRImWgXYmLO7quacFIDz8nOnV82dzEks0Q0wr12/A6ee4E2\n6m2m0x9SV8fous8/ZBTJ6y/cjamzGFq8y87UrL79fe7e9OaTLDvRMIVJd55QAtVVtMF7QZm0buD8\n9cVnLDnywOPP45Rj6O/52wvc51hFSzU3SURZTTECaY6FqTMZvt3cRu0uFefzqaqidtexbgOCBZRp\nfyK3KVozdA0NDY08wVZl6Bu6aHPbcRZXqP5YMpvcEhf2kDa5in2xTMrUrl6Hv13O3W7OvoB2rrIb\naKtdK63PJod4DCTFVt51DT3StVJk692HGLdspkPo6uaqOa6BxfE/X8hdQXbZhd5xJxZDICCJGVLI\nKRlfBwAor2Cgf0c72clfnrwN518wl+e5aseVkYtVFQjLxpi7brakblYDGWabL046mN/AWG2fsO+I\nMNaQnNsj9vdOoxAJYaG2pCx7VMy7fJ+CFx57aEiN2tAgpuKFXQMh6Z+bJg1NLeazqhtDG3VaIpZC\nxQXo2UA/ydrVTJConzRyhu6NUku7tpWf/3x9AyaIbfqjt3m9Cd9lokqzFCH7oCGE5gLK3XYUe5fo\nJ2HxpipqBTNLBpX8VTSUirTweb1wlUykNG7MpKzHP8XEorTrwBKmOaOM0SxSmQC1DWzvqn62d+bE\ncSiYQLkFmoYWERsp1LhVuRuW5WRLZ3g8skmL+I66u/hson3xLIsPhtn+YED2BM3wfXRlpy7bNOBI\nxIsacwmJGrNlExgPfEgqmZm0/Qb8quSF+CWQgk+V+pU6aKatyvKqTVKkMJ4LGOLcScdy90GNq6Um\n1NjF2O7XX3wPy6VIXIkk5YwupF175QpGZx176iUoHkV7esvn3CXowl8ycu6wg48EACz8irsINUzc\nHbVVHGNrG6nR2zFxOoisT5pzCg77Hv18nhrOG289ydwWJyXzSMcazO/8BAAweQajXXbbhffa2EXG\nX1jA8dEXNdHazfj/+uranOShGbqGhoZGnmCrMvSUeG4/W0C715TJkzCmnlu6NbWS8cYzXA3dGFlD\n9ajJCEsM7FUXc6fuPbdnWv7BhzNNXZnwenv78Lc/8pjKR2QXdynXuWDBagDA7jtWICaFgda30E6+\n606M1ujpJp2oKg+jtZMMs2YUPdnRKBnhhrVspyn24tNPvwQZKYYPM/fynwOPYMCIroowKbju0Ot2\n+mzUzqa2UigRE+mMYmq8XipBBv/TW59GbRH70NNDhr12FUsvvHnjPwAAP/rJ2ehbR5mMbaAtMi5x\nxAGJNe5s60dRuWRFmpRpopwyKpaYfJWtmMykUF3KdowvGTkzV2hrY0naccLmxk4bhafuZiTNvocy\nUirQzHtW9ZDdnR4rwPo1HDvTd1MlUNn2RYvJ3szRHEfvFEexMils3lARRbIHpzzD3r4oIrIPpoqT\nznlxIEQAACAASURBVBZ+E2ZmO0bWf1NTzz9ChfQnfDmf5YO3r2H00bi6ctxyN7cuO3jHH+csEwDo\nkZK2pjGwz6drK79AWtrKMWyKhmV7BsoQ9ElkiWPw2KJSv/RdFf2ys2ntjnynshhdOTfpOOiK8R7+\nMOVsxChnSzav4E1V/V5h/NJm9XVG3ccADNWfLZDJ+HG7AwCa5zFCbcx2EdTtwNT6UaVse1+S19/v\nMPrIzj3uIEwezVIgn61jbsztd9InZaXYh7SXY6akfgra1zcCALySp5K2RdZRakyjysMwLL4LlQWM\n4z/0MGaI/vGv3LYwGDAQDvHdWrecWtvKRjLzg49guz58k1nfCX8AZbJJTiYdyUkeW9cpKpu3zjhw\nDwCACy9WG3x5165n5TE1l40eywm0rz2ORJrNrK+n8/KxRxkKdNt1FFbZKDo9PFYA777DdPFR1Xzx\nJ02iA2r6FEkugINQEQfi6DCvGwyJ8ytMlcd0UwhYHOw9PZwkikSlzjh0YBQV8GU3TROulC1IxvnC\nFWHkeyPaWY1fqkMaLtzs0Fa10oeGdvptICoDqDTGAaheFDepzD5SG6c4gF4JL/OJcFvXcceiP//u\nRgDA9759pPJnIbqR1w1H2Rd/iC9vZdDAV58wLXzcBF5P7SVpSvXBjHjQQt4QVqx6FwAw/z3WOjn6\nwjtGKBFgxh589i/d3wgA2FD8ESbPYF3oytF8Lp9LWODuezYAAP7w+09RX0W533Yn+37smTQ77Lkb\nnWNTd+EuMM+98Dds085+rlzGRd1s4/OtrOZY2PXwKbhrPh3NvbM4dmIeNfmL6cZFdhPPv8/mvQ+5\nn+Nj0jY0uWzYwPorC5etx08OZJJJd298xLIYDEcGy+Adq9SfyjzkOGrHLzE7IZWt351MiBktrWqS\nU5b+oKrYaAIpSV7yyLsAhjyaKU4sPf0OkhKa51jZ7Zh4PTHpwXHgKLOeR8ajTOxqIs+Ik9X0WNnw\nXNPKvdriikaaUUZP4AT98XvvYbvtmPJfMo6Oz7ce4e5E5fWs2HnZjQ8BvTSbnFxBsqP2+0RY+pZk\nf1/825/QMI7XrisXR6VyvksJgOtu+zVuq+HzblpDE07DOM5xZ57DciUtvRtQV8pJ2i+ymSzhzk3r\nGgEA6RSvN2F8HTxpmlxam3PbOFubXDQ0NDTyBFuVobd2kDFVjuJK1dnVicZGMvOAqLcHH0AVSjk6\n3/p4EaaPpdOtU4L3TzyOVcuefokmA9NLxt0XXYI33maVv2/vT+ZUKyadsfVk7D6PBxlhKG+9x0D+\n7afTeTKqmswQPi8+m8+QoqA4VOrF5FI/poHHSMJNy/pWhMSkceX1LBr2p7tvzUEqKqSOGLxz0aZl\nAYiUa8ArxblsyAqeDXHkhyXqoR/9aFfFmIrIQk1J74+IRjJmbAOiMTKx0mJ+5/GqeuES/gkXtXV0\nIqtiZFVV7LcySSjHbnd3Dwwfk0ym7XbMvxbBMKxZQfa98x6sBtieKYTVR3b0/MN0Trd0k4v85cEP\nAAB7z6zHm+8wRPKYg3jeS/dSJd59Bsdb/TY0/7S2dyPaw9+q/dTgVnfSDLX9gWTz4YSJmof57Ke/\nznDDvb7LXYiCwmyNAiDtpfw3xPjcPl9B5h8ooOYZlBIUXv9yLF7C9pVIQkquMI1hyV8AII5etYtR\ntqZ49ns3a45R5TVSaZqe2jaKWUUcqplMGs2NHAeOX5J9pNTC6uV89yxPBLb4dNMmtRG/zet5HFVT\n3M063BX7Vs5mQ5m21G5hjidbPC1rv8oFhXSKGinKe7e9v4NqqcC5/DPWpD/95ywjMv/d1wAA6zqj\nePU1BkkcdwpDodc20wxSN55zgWmTwQdChagby7kjmGZCXdrh/OFKctsJP/lhNszXH2kAABQVyt4A\nYg927AQQENOplDLpTNIEE5LEovc/4vxxeM1sLFnA+WfWnofnJA7N0DU0NDTyBMamtbc1NDQ0NP5f\nhGboGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGR\nJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho\n5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoa\nGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaG\nhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6h\noaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFr\naGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/o\nGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9AT\nuoaGhkaeQE/oGhoaGnkCPaFraGho5An0hK6hoaGRJ9ATuoaGhkaewLM1b2ZZlgsAruv+W9cxDAP/\niev8t+A4jjHSY886+WgXAJLJJADAcgHTdAAAhYWFQ461bRsAEEvZ6OvrAwBkMpnNXtfn8wEA/H4/\nTHPouq3OcRzeJ+na2e+SSbmeaw45tre3F8FgEADQ09MDAPB4OHw8XndIe/1+f/ZYde97739mxDKJ\nttsuAHjDcQBAX5ONZSuaAABlJREAQPW4YgBAKpUGAHS3m/CHKJ9AJAAAsG3ee93qDQCASKgIADCq\nrgAev8V+9XWynyk/r9PWz/tUBxCPUz4eg/20bHYhlqJMvN4MfF4vrx1hu8IhXnfhl7yn7adsggjB\na6blnikAwE77jxuxTABg4YIuFwCqa0oAAKZloLe7g/KoLuX9HMogmWTbfT7A62Ob+mO87+fzFgAA\ntt9uRwBATw/7/MnHn2HHWTPY3hBl197GcbZw/mJez1ON+tETKI8A+7Zq1Ur2q4MyCPjq4Zd+pzOU\nw7f2HwsAaNrA5zh9Rh3bmYkhnWa7mtZzXE3ddvSI5WLcd86/nARcY/jlXACbv4WxybG54evmJGPI\n1+43Huu6LlQz1CHGiXeOqGGaoWtoaGjkCbYqQ/9PMer/VWa+JXCEYRo2GZUDQIgz0mn2UzFp9WnY\nTpb5eqyhTFjB6yFzdGzAFIapzofLxd7O8N6mA3hEqbClHYqZp20e4/f7kUgkpF1ynknmV+AjMw97\nyNA8hgdGiu3x+f05SgTIuLxP66ouAEDAH0ZddSUAwLLZLg9475YOsnifL4CgPwwA6Gwh6/Z52Z5g\nmJ8lowoAANG+NNzeXra9gN+t2dAKAEgmB7S/SAFl290Wl35Rpql+tsEKuognogAAJ0WWuW41n1lp\nmIzZ9YvMvDbSKT6HYs+WvXa9/THefy21ufETXFRU8LdXXn0dADBxwlQAQHUN5eW4TvZZWhbvO2HC\nJADAggWrAAB1NeMAADNm7IKyUvY5FuMzLoxQ09t+5iwAgGlZSKYo+6IKPttQWLFvyqlpXRdcg1rS\n2AYy8Q/fWcN2iSah3mGvx4Fp8u/auoqcZTJ8LjAMI/ud+jTM4bzVQJYlD2Pkm5zjOIjKuArG2D/T\ns+n8s0k7sv+6mxzzdVrAkN/V35s98uuhGbqGhoZGnmCrMvT/NgzDO2BfN2hLdBUrVWuXYZMG/48g\ny7Q9w1g0gHg8PuQ7daxpmvB6QvzN3HxnhjP2wVAMeyTnKc0hnUmjvz8h34rt3CGLSAk7HdxO9Ry+\nqR1fh6CwwmA3mX9pRRGam8jWe2O8XvcatiUV5f+l5Sl0dtKeHAyRHZaVkW0Ge0TLAJltjxFDOs57\nxHvJeosiZOp9Is/e3l6EI+yf5WP/ojGyeicjtv2uYlSOol1+YzNtxT4/750JsP+Ran5aXQVIx4Vh\nx6I5ywQAOjrEhxKnBtLb72Dm1FEAgG8fuA/b1C3M1yvjyU1ln09SBr4/JH2NrQcArG+mdgIjhZRN\n1p0UuXZ08rfddyfzb+voRtqhHCHHLv2qHQAwduwYyiC4AevX8Nq2UQ0AmDhpNABgQyP7vnxJGwCg\nvNqDomJeJxD896cj13U3YcKbavRu9jdDsfksFZZz6MbBc55J8B95FgCgzWC/u5+6EQBQkOJ4szJG\nlkm7wyj15qwJX2thGPR9VhtX7dz8GZvgf39CH9COsr1Sqq8jE8o2l14MACg48ijERK105GBDdB8r\nQwGVOim8P+dEAEB62SIe43Dwus7mHYz/TViWNeRTOT7/FQZP7sDQheDrvh9+TMbiROcYDpAxhhyT\nkpc/lmJ7otE4eqOcGII+HpuURSgU4OQ4WNX1i6lF9SsXdLRwwiyqpuOzt6cbKZv3Chfy2Xd0clIb\nN4XmlIxjIRSnszCe4vlpmXg3NnHCKYrwd1gWwuVUo2Nd3QAAjziRLYcLhc8XAWRcpOJcGHyGfJZx\nQoRhoKCC7em22dZAmv8nZPIulu8NDxCzRX61OYsEALDzTvUAgNbWcgDA/C9bsGH1ZwCAgw/lZNpv\nUx5eMXkZQSCaYpua1/XKlfjcZ207GQCQtOn4DAVDMC0uSAmLz7h/AeX+/oec2CvLfRhbz0VsdTvl\nW1heBgBo76VzdVRNLcIF/K55w0YAgMcnz7SCMly5mG0xwoVYuIbPZ9tKTv6VDSOXyfCJbzC+ycSx\nyaRqyLOWGfm3qzkezrj6DFwaYNtdi3K86Yecbx5/5m4AQKZ3IQxDTIuuGu9fbxb+WsfpoHYOLEbS\nv6+92lBok4uGhoZGnuB/n6G7A6uVcup89/0PAQCrvTQ7CLlEL1x41QopJhdbOQCFKbaZYUx68FEe\nI47ANVdcCgCIvvIsb7kVna4DFokBxj2cSSts7vuvM2kMZsbDz1O/FXlo0ii49AIsvejCIcfaYpZR\nZp/BZposixdVvkvYcsGoGgAD5qNvat83ob2d9+rvIuMzQ3GUlbGtTRupqhdLmKBtk1H6QyEsW/EF\nAKB5PUPs0hmenxT1uUBCKSORACIpOg2Lg2TbXpOfZcWUTTDoQTTKfnUlGU4X9FcBAEJhOu9SbhrN\nXWTiXpd97pD/IxVKkxCHasLB6HGk5nEo01VucMExXVxK9pix02jpYP8//aoFAFBVxvtCHNZ966Lo\n7qDz019Js4nroRw+WELWHRATVyIWhcehfKdP5XVLRR7121FeHj/QupFyXfjJEgDAdttNBwDMm/cp\nAKAyMAurl34FAPjWAXSmLlhME4wlWvD42gYAwHvvNyPjIbtd/Qn7cPLZ1bkLZxCGv7+bY+rDWbsN\ntuE+H7WWM+fOAQBcN9OA4/A3O035X/Uk5XXozucCAO5Y0YY5zbcDAErS3XLPzTtb/9V3m7YtN7eo\nZugaGhoaeYL/fYYOExDb55Q3PwAArLSUrduVI/i/AQu2IY65Yb5Q9YdjZrLWrf+vvesMsKuq1t9p\nt8+dnjaTZBJIDxBKEgyhWKhGBEHeUywoTRFURBQFBQERQSlPER+WJ1jRRwcFRIWAQGgppEB6TyaT\n6eWW096Pb+1zpyQ4E3mPvHGvP3fmlnP22eecfb611re+Zcl7E274DgAgdfW3AACvHD0HgcTcQ/zv\nxtUTgmA9U8Wwjb3Gxfdk6juOFLjsyfaGkp3rrwUAFBAikWFctr2ZyT14arsquenAsgS9S5xfoXZV\nwNNVIBotKyuLxrMvCN2OEwFmJbmZrsoiXySqra5gbDYs8izu2ESUWVbRg2efpefV0sPfd9iMk7tS\nbBMTdJgAUAzoXTgyPFPokKZP9GWEJsyAxxO3OQeTDp0CADhq3tkAgFy3iUIrPxtbp+LsQo3MNcn2\n6FmkRjtoaW2X33E8VZXJIc1LZzuTchKexuFzJ+J7d/Ac3nA3EXHeeJPjcHkc1eZIxM31cpCkEFaN\np6cwopYeR00Fx5jv8tBQT0rj1k2850ZUHQoAaGkm8hw1LomWLs79uka+Nz3JOZj9vnncT+AiIxTE\nra08b57NOPnalTyGCaPpyRx4QAc6JccaGkP3jAdQFHtT/lQcWr6rrlvTNAFTdirI/B7nJADAx846\nGgBw2YO81pOeDbV2GLK25GS7v3mVW7ZDG82nXsND/+8vyz4VKaM0Psvqe1+/VWHRvppG6Nq0adM2\nTGy/RejyMIOfqcD0ZxYCKLFaDCnBDuQpWPa3pwEAL19xaUTo7/+UM00equ8kcMJf+f1tws7wZRq6\nkvx/5ouvYulsxgVNiYUF0YbflsOLTCFZS8brWdZemS59kLs6c+GekfCe0L1CC809RE3lUEgmjuwN\n3+Bn550LAMgJIrZlM0YAFPttTxWsqLnukXi7afxzOOGX/3UBAGBkFeOZC06/CHlhmtTXEV0WHLIk\nxo4iir/1umvQKQUgUz53KQCgcbTQSeR6sWxhM3k+HFPR0+hlKKRuh+KRBQYMEEXml7NUfu2Dj/A4\nu38NAJh31BnoKjDGvOYNegXxNK+h8gqOt6KSCDCZTmJ3z055L7Mv04IdLRzrijVE/xbS+P0v7wIA\nzPgy49lrKukFZCRO7jTn0Cz5hmSZUD3z/M5WCc8WCizHL3a3A51E0vYrGwAAZstzAACjm56Q566F\n10W6XryJKPf223kubMlblaVSqEjTU5nQwKKj1NhDAABV1Q0AgG0FeoRTxs1EVs4LpBhrKDagoAcD\nab7YA4U2dHlebo0tAAB84t/fDQC46iF+JyP3YGiFEeLvj/RtkccIzXLc8Tg/u+mT9PZfvednAIDZ\nJs+LEeydwdafmtj7mIaK1jVC16ZNm7ZhYvstQg+ltHvawoVRxtiReHaFPCufP/IwAEAgJewh9g6g\nfUFilu/hyaOO4JvCcJi2kKwZ9Yh0LRNzF5ExsfhdzNK76on9NlclRQye3hl4m8eudLf2VCChnsRB\nqNgnjhwCxxlgIMvFDfnZhNt+Ku8TVcIG7BhR3JgxRFRLVy6Wj3rz0/s9/8WNSgvnfHQ146Yhwn2K\nnSu74fwZAIBOpwEAcP/CxzH/8LO4bVvi0B4R5I9v+SEAoLE6i1mXfpO/s4mS00UZuyMxS5lQP7QQ\neoKWZG7dQH0mt0QQwjQ4h8Z0XgNlo4gqV/4H+ced7T/BScf/OwAgk2U82vV5TbUJuyXXSJSfRA+q\nRpO/3R0VFpUNaV62befvJk44AACwesUWXHDuxQCAv+Qe4LYnzAEA5IVtkakqoCjejOty/AWT59o0\nJPeUYNzcyBThj+C483K/2IJk4x6vLyfMI9Yj7LICWUDZmLou+RvX60SrMNLa5NrLFATN7hJZhi0U\n9Fr5vW9i47N/4XYy++a5AKV7IwgHXnsDi3QC3FL7CQDAeR+g53D1g5yL206mH+pZUtRnxrCmjZ/d\n+RTHHk+IRIHklEyrDX6eLKArHuXvvrSAxUi/fp3ezMfX34rAaJPx+H3GE3kUve7zfY2j7z8LulrP\nhG510AukQBmuBdfmhI7ZxSKFx085sc9Ph3LoPkKouq5QKv/WzeODYfILL/N9w0GXFJocfvN3AQAv\nXP4ltYG31RSFUJ1UG4Apf1sS5ilAXQC96ICeXHhy7YZh3/AHDBUr8eFJQYthlcl2ZdG3Ga6IuQFM\noRq6V17O3539cW5Pngt+GETzlhE1w5QsnEppMAofvQVlcjDmFrndIGRowegZg4zD95a/Rs2SJ5/9\nEwCgvYoL6ayLv4g2pWsjIRYjCqtIqEpRycJQ1QyVTMJEah6tMIRnqgQpf9ddRTrdlM9dAgBY/eP/\nxP2/Y8jjyJM/BgCorWwAAKQyqtCEc2/5DopyzsIwNaT5UNaV53wU5aERTwU46uDLAAA/vZyVmLO+\n8BsAwLbR0wAAhd3rkf8pk7hVGR6jk2KBUnlKFvYYs6yp7EjEKrnoWxWiBZPkZ3lRVvSSLrrj/F0h\nkASwrzSHJOloJOEIoLBkcd2e4rURTzMsZoPXEFI1WLuc9/WunUzenvHp2YOek6gSXJ1aGPD8vgum\nJbdEIPu8b8z5OP8DnJ9r75PqWbm3/usljuWco/ldK7Axo4rHchyeAQB8aMGH+JnL7V/ySDVsAXpu\nka/f/QvPdULUN19/z1WY9uTlMicCIlTRUDQ+OYYwHLKGizIdctGmTZu2YWL7D0IXc2bx6VwURGaG\nQKUoEv5lwckA/vnCn/6/dwU5rZhLfeipL78W0R47j3sfxyUhDRd710F5u025jnEZi+/30jFXGjAh\nE1PFsC+lUJ3a0POi96q/eyvkTQBA7ld/BADsXvsIqq79MQCgPMHE1NSJMwEAa9cvi8aiNl2RFN3r\nJFFMfx2aIAgiSuO+lP7bUfm0eC92AX97mZTEN9YyyVQ2h+el+gR6ay1OGVIFnqOCLUhYthL6Cvso\nOQgD4T/w63qr9vkWDzwlyHrHKHoCY75yKTbccgsA4IU/3gMAOO3siwAAmbQUWTlEsfkeA+27BbUl\n9+36HTGGrn5bO5ObY+rHIm/Txf/9938PALjwAXowxZOo8RJUJhBr4D0VW/wQx28yXGWJCqXXw/O3\nybDgSOrbkTBmINe9Y6iUeABXhffkmrMCScynGVKqcrqQhxQ/zac3kz/gvQCApGTZ4y/+FQDwpbOO\nwQGH0guobB265xKoe1k5pr2grTp/rpTs/7GaXtSxJ07Dt+7lOfUKKhvO/1e2iJchejVeEMCS0Nux\nxx0DOWAAwKWP0tsod4But28i1pDfFGRu71rs4j/Opbf/3N23AQCOCDZz1+pYeoWGlJdhD/H+0Qhd\nmzZt2oaJ7TcIXdEDp/7sbgCALwk8Lwzw96MkpqZiy28hZFV6Qu9Zac00TZx1FhNsr7xGYaN1a9bK\n9vmy8XOfw/gf3QEAsH0+3U/8A5NOj52xQLY79GPckxmCBNRzOAiCKB4Y+BKHjLwCpWvtI5A4sVIQ\nNOXZXBC05HlCOwTgmkSUoSlJMFf2ueVlOcasAihwJQ4++TomGNd84qOyfQ+eFBsp1b6UoBF1PhQ6\ncV03+ntfELphcOwqkrh+y5sIpTy8+owzuK9pTP4VlWBbMUBRxMYgyDFQSTCJj3tR4UqI0OQBx12h\ns5oqvh1Gm1Ba2ZZSlVR5Duk81BVPYfJXvwgAWHHLDwAA997NEvBLrriS3xHEF5hZpMo4t3aYk3EM\nLQn45ioi6zmHklK7ctkKjJ1MT6BYQRGw8GVSfIvHsVCmpqwKFad+BgCwahk/qzIpxpVvYVLcEtmD\nhGPAkkSpL8foSQ4glNyWHRqISaxc3X85USHM5ri9nYUsECNaz9bxPHkSZ0/2ME7etOJRAMBXfrkE\n27cxl+XUvB1qi0GJNisifvelPgwAmHcyyRA3PFhEKF6GJUVjAqgRqmuvwOvNjAeRjEhVldAqA375\nqvfz/e8/WYg8D+WtdnZKIl7i5aYfw5ce4HuXfZziXne/ynGe88Z5/E6x5Omq+H9EkRzk8WuErk2b\nNm3DxN55hK4agzh86oWBdPARlkbDrka8GUTVQgCA11cs/4ebnT51Wp//1RNvxaqVe+xyAgAzprGQ\nxX3pBdiq0EeC6WsnkM6n4mle+PbQXQJBQIEwMUwrgKE8ETk9MVMQnSCPpONFdML2okLFgogFzQdW\niV1SdwOZGJ6IQuWfpYTCjh3M6DuOg8JnieJq72IsvV3iq+PqyXbYtqMRsZjE5QWhRXTIqIPRQIEx\nJeA1JFNulkIpoYlZXyAS3p4lEi3I/MeFbuj4ITrlGlJA3bSIGF1TIW3xVIwQMaEwFo20vEd6oa0Y\nLbaPVJ4xXT9OuqAniC9e5HbiZoAWOW8TPv1ZAMDqX3Oub7+JHs5XrmLc1Ms7sEP+Lsjtm3s3ehTj\n4jnpmNTR3o5sSCS8qYryuTfdxv2e813KWDSd/kO0ljNGXfb+03g895MJkw15jnclibDTbg6OIOm8\nIXLC4tU5QcnT8sS7CUUcLysoMukR+Xf5LrIfo+CdEec1U+VyDnc+eC8AYMPiFwEABfRg4gQWgPVA\neS5DMFVGryiACCgHDeD+0ecDAE46niy2a+4TGQLDiRhkQT/amqIPezFSDC2kEAbixYkUhCPeSoV4\nzNefVMA3H+Pvcj19GT9hFFM3oFoN3/owP8vESWlcfyyjAaMXkrEUz7sww755n8GaRujatGnTNkzs\nHUfoqrnEcXf9EgCwS6LJioXwxIIFEWKN+lz0I+QvfPqZ0vYMo8+rQuPLV66I/j9oBhkcX/wiUd+n\nzmcMa/x4Fmxs3Lge1YsYX981Z3bffZqq8OTtQehhgchQPYfjhlkqCpLYXXVCRLCEdG47AWIyG4pj\nO0r4rq1SsyLtSLEjXw3Pkr6WeaLlxgeIklQ/0kTCRk7iuobbV9XskCu/DwBo+tKnSywWxR4QZK74\n53tqpjHYhh29TUL1kextaNpoK5OYprAw/L/+meP6+0uyPx/JafSiKt9/PADgzVt/zt+HIsMrl7tr\nhXCOYBx67Hs+yN8/ypjuunVk0Uw++miEM2YBKDW2aHuO+7QkFo64g6oJ/E6+jEh52oUsKNl807cB\nAI88+FsAwEmnnIvOApGwoXIQQ5yXjMR7/SKR4UEHT0OPxOHrZdKWdsh5cyi4tcNoR75HhNaOJjOo\n9alF/D/cAgCw5ZzbQQDfUD015fqSa9Ev1aUDUvLuSy6m6KsuYcL0mPIutMbp2cXl9Fdv5T6PmMWY\nv1sUz8jPYU0ThcVGptnbFBWDn5Me8RqXr2fB2aUPL8EJl/0KAHDZArLWrv2t5EDEwzLDMPL+7EgO\nQm4YufWuf4hx/Ws/VIWv/IU31WRhFJ0zX1g+4jEbYQILDuI4fv+yiLyZck+ogfbilvtSpNUWkLX0\nw6W8P7//kesBAAvvewxzWskCSgxRRkMjdG3atGkbJvaOI3Qlmdk5TcT3owpHKbv2i3tlDNtScZs0\nkAAAIABJREFUXnzRReT+hggHlI32R+ymYUao8RbhEJ93AQWpnvgzFXamTJqMhV+4EAAw7cXFfbbn\nTGVZemH5a0M4yr1bKkaUkzLVU9tFlc2/ExIM9mQu0tL6ra3bBwhwkE0q2VxpfpDh/EmYFakrbwYE\n6Y5oYszOrWR1pWKg7GzcgriU/q+7hLH0hh8xlt5aSxlVEw7iEpcPBNXkVcs+FbPuFUNXwl2FopIp\nHbwZwkNPqmvAiiEn14npEm33PEeGzqgTyUPPjK3Hph+TYWLPJbPioAu+AgBofPG/AQDFV3guK848\nH4Vy4uOtd/I3zk4islpB5bvvewLxdpZ6Z46kcFP73/4OAOiW85PzczCL9wEAZlxzFQCgQ3q9Fiyp\noxCPrtgdIBC5AcMcOvMHAGoruM1tO9hfNVGWxbZG5kGqAnp640cdCAC49mv0rE773IXIfpZeg1tg\nNem4M9mCcesvGG8vF2ZUjxOgVArdr8K3n0wtANjSpq4Yk9+bDQCAzMmXIpYizK4rEjmvfZze0vNL\n6fk2NlIEbExDHUZlJFeUV40/4oOZDgDARffRo9rcyP2cdtEtOPej9MC/8RvlUfK7vQlwgdwTOalx\nsWNK+I/XcI/DefjaH7aiaHE8q+Vc3ruI8fWPzq2U7bs4fCx/d/9ionhPvB5DCeqFQem8q9J/GVfC\nowd49Z/k9YwFMH5BhB55DoO0d3xBVytwt7gW4ulFya7eBSADNE1kRlasYocay7Jww/V0de++++5e\nWy+Z2avyoL+Wch/1wMKeC4iO/hoXiafO/vfBHuBbmhMqtUWGEsx8ACultMjF7ZV7zJTJqUr5cBWl\n0eybkFQpyFe38OavQ4i8JHa330MNkkKRi+LNN98cjeOrX+GCVDWCbqAhvTWDgNsZe/qZ2P3o/W95\nLCoB6nletOh3dHS81U/2Yn0fwoABW3zhuCSoqiRclJIwy/ayDPyTmPTrChluKFbzIeWWS4m6OKSZ\naRNhyZwYTVwIEjOZRE+efjoAYFfnDnQuYvI9dTR1vm05vpqzTgAAhFMOxo5vsmHwrjcYNojN4ANf\nJcVUoye3x0cqLb1Y8/u2oE+fzpL9puY13Jdj4eB6oS1mObYqk4tPIEnej8w/GkukZ+gai98Npzfw\nVUKMzmpuz7cBmKqgT86/LNpxSb7nzWJ0D1keH9b5Av8f8REmhnfGa1Ep4GPL/XcCAG773k0AgO4c\n77FsuSe/9WGJhECirHPIc3L8wUz49hzI7kEf/sgJ+Oa9AgRMFfZTBTuQVyOiF5oCIAP5To+nkuxC\ndXXqYSuCgej8vLFFxnmkWodMGLJeKRJFlNJUMRAfpYU8CisLNVtWqSuP53z+9ZHnMV/07O1gaN2t\ndMhFmzZt2oaJ7QcIXRH7hRInT8xkcWCnoAG9AeX12WefBQAcc8wxuOLrpEvdcw9Lsfsrrxl7UAFU\nao5KXa/XpiO3SCWFkvXjBnlcgzPflSdwrIRKPUN5ChxrTnoZZpKqvNiCH/Ug5c9VEjIp8zni5h9F\n748SjeulOxhC+MEdt8j2peOKbePGG4k0L/sqPZCubxO911/JApnkB09D4ZE/cBwxlvyr0FVEURRc\nYoYmArefyz4kk5L7Xip0ti8FGwm+t1ma/di381jGHPNuZOczNBKYgtqEApuQprM9SkffN6PEXs7m\n/KctIlpF++xCJ5LiGaniLSUBIF+FWQhgCFWyUihpeZTLIRApxxxVuBRDUwd7Zno5hmVqh1hY5IuH\nVl7BfRiIY/VmJvtnTCfaNiVUIPlKnPbpD+K3l5C+53yBXojjMfk47jyGXJq/w5ClEbQiJyX7WU96\nksp1nxV02mMVIrc3IUm9rmlMPnZPozRH2u/C+LWkJabreL4WnPJ+bq+V13ZtNT0rF3lAJC268pyz\nRHbwczJ/JEM7rzWwm9RIK0CPyGEkAxX26usRmaYZhVyCfmuKuu89JRcRhvCFeWALXdOKqY5DJbKG\nolkrEc+oK5ps3zSMKPwQyn1jiQroDaLw+NATTByfvOO/4ap1MRzafaQRujZt2rQNE3vHEbpCwIpI\nL2AKRRXj6lWkomzalKl73pZh4HWJp7+xmsmSqZP7fXcPPP3o6WyWIu79mnZHTz77be5Y1N4j5fgi\nRVsR96K8VCheS3M3d7qxnR/MGAnEItAhhUVS+PHwBs7VhF5TtkryCqr0PfJaJH7v+y5CSUR2dTG+\nXmGXxL0AIgxnPBPXdtPmPtsp0RkHUhT3qfTfUnIILLU2zCJiSigrIPpu+CJj/ljB5PSqR+5H2VMU\nppr6dXoVeZtyt0WZIzVnoW3AFJQUC0WqV3V2lzGU5UyEBhNcBSUvICi1/Snqd2PhC0gqKdSJTEb2\nSOl4RrwBv0vJNYRIyvHEnX0TR92wjkiuvZ3nKAx6kJMuUe1t3O+b2zYCACZObgAAeEUbN1zCDk5X\nLya915/6LgDAzjiRfjiONM/EiodgJqUMX/VWlURgR4z7zgQGYNFD2ynI+pAzSfvdWqAnaBgFLL2f\n1Njla54GABS62K2pZjThd+BLkY6TQU+O109FZvDJUGXl4oXNXUXphXu2zsXdH/kIAODiB6SUXr4r\nemMIPA921PFL3lNcjOje8KPfKvlkhd59sy8RIQzzpb4Bso6phHFp7fKjgTgiB37dqfSYH/wTr+ET\ntzBHlQ+BeLRt3bFImzZt2v4l7R1H6IgKZFR2WGhDkWD94J9QhmH0Kjrac3l/4A+MSfUXwqHJ013F\n0uXdlQ8+POjxDMbWtYv4URNfkwkbtdXc93iJ2aVEbnW3CP40d9uoTvOz9pzqRsQRjv9PlnWb3RI/\n7uiIjqumol/PRqOXxG1YKlEGgJ4eBorHCQOiacpkTLqSqHjbFxiTjUunokhAqJcnpSiN+9K5SFH9\nXFd5TsWonNvME51mXY4vJzTSmdMPxIYbWZixeynpieERjPG6kdiZuiaCKF+iyrFV0ZHjSOOF0IQb\ntRcVVClosLyK8eqmqlo0vOc4AEBXhkwNJSymisJGjGKMN52JobOdv+/J0wOpHsqkAGhtZoFLKN5Y\nZWUZxo6l4FRHB7dpO2xQsWr5Nh6X5eEDJxCx3nYtu/P48+hpFfN8rfkEaY1dl7+AfGw1ACCRExaV\nyFi7Eu9N9viIW+TMVopcxDqX81wMmCOoevZR3PUDSg8420ixTI7gPHR3SXw7xm00NuXQ0kSv4NVd\nZBydeuaMQc9JTuLbcbkejun6Cx67ifH7b11K1H7To6QZtlvC4PLNXutE3+s0apjRy5VXa4lC+Jac\nY1dK/01EzGB0+UzulCs436vhjIB2XP1BkdT9OZu0HL6dnuVuyZFZMQc9sk/VKa1ykPOhEbo2bdq0\nDRN75xG6PLVqfT65d4jUqwHV/quUMX7iiScAAOMaKESkBLh688kNlbVWT9x+MSjPcwfEfqN+hL1j\n9fWjeg8vsjd//pOhH+NbWJNk/VVv0LgbR7cgw6IkFEZK264qaTKwuduFJwigM8fXlV38zoTuvvz5\nbTdfFxVQDZC0DVUs0I3Q+p0/VE0wuN1rrmMX8/JLrkSPxFGLFfUAgIyneiQGA16DsO8cD8V8pVsg\nsUbHtBFIj0cUiea23cU6g4YLzgEAtFSmkVW9Z82+jS4MuaYsYcr4hgWIXG5a0KYpcWEvIY0JLBM9\nKqIu24vLHAXHHQwAyB5wJDrBbfsWuckJOd5dqilGWuL3SQ9xaaaQK6h5H5r1FHfxWNsZj542cy5S\nSYnhSwVCWtB/XQPxfzKVx+5m5pUuv5LNSq78I2O1iXlE80WRJq458TRseZU9WhXrzAoV20zK/Z00\ndpfx/JeNP4rvuZyzcdtY5BTsbsJxM8h8Mau47bxcpylpd7d2zSYAQKY8jpmHjJF91gx5Tl5vIVp+\ndQuL5jwvwKyJFPuy7/4EAGDW9OsAACs3Urq3OXYYXKGjWNaer1Nb+vrG4oCqjXOV3IcpXpxJr8D0\ne3XQFOG1MNl3bTGcFtz4Ac7pkw9SxvikZspWxMo5R6EreQWU2icmjaHloN7xBV0tt89dSPdt8s+o\nw+BLuCEYOw6xzTz5J59MWpTSZVkpxRyLX2VS4dDDD4u2O3M63baoGEmFWgwjKkRSJWSKunTwIXRJ\nbTuOuX9gaEUtrnk1U13N/8TRDrRcrm8lZSGfixJdPT18qLVmeNGOreLiEwRBpJWeEH2Putup2CZe\nIHp8uueGYSAm/VH7F2YZxsDiKU/iDGvWsIlvWwsTN7XxNhREN/vA668FALR8k+pwjj9Qt0W5svuy\noJuymCAanw27KP1P4yyuaSsj9W/dHfKADbvRFZAGOHY6C4G2CW3NtzkXPXEed1kQIOlyLpsnkcLn\nvc5G4c0/Y8ihfOcupOeRBhmXIo8dogVSL9V7rYYJU5JnvuqypFQJJdmak/PU2V1ALM5j6AhUk+ih\n2ZzZTLxmykjR27ChE8UuHnNQZDjNL0oCfT0fttVZwJGH64nzjwQA3PIDVpEmj9oOAFhnc0HNnPJ+\nFF7hdV80+JkjD7WMxECbyjKY8nkmnXcmCaxq868DANbeS6XJFx57MNLOTyV4zIFS+izywTdxIhPW\nhg2E8jCybFUWlxj0nBxSye8eXCFFU6EZZS83tnGeJy1kUnjLbIKA4w/M4dcv81ru30Ba/e8p/iGM\nqFOXI/ea6gVq++ph5yKQlSwjRWMi1wI7ybm/9aRW3PcQF/JJmx8BACTjSoNfQi2SqfV9H7bS8B9i\nhzQdctGmTZu2YWLvOEJXEL3w+hIAgG/0ZeYf+sDDWHoEkbd6ks07krSr55+nrvdhRxwebW7WwUTZ\nqoxf2QwpyV6zZg085TpJluLLorWtEiI+ArTF5REr72VVkYsgsf46yvtqrvIcBNiZpom4QResVRJd\nParISjREJtTYUTf1Za10YesClUAURcabiMKKxSI+8xl6PyNGUAHvuuuu2+t4vnz5lwEAnhQz1dQw\n9LTq8q9j9K3UPbFkX65Q25KxviEXz/MQd/beVeofmQq5WKJL43tAVjRBmpKk2h14HlFXrJ3ejBka\n8Cs4Fy1Sqh2XApPsFCLtdBXRuBumEMh1VnkW3fL46wyjbHqBbnDtv50GTKeuSxAjSmodSe+gOc05\nMeEgVFw0RX906dFYcn0kqulJJMtsmBK+GDV+aAVFyl5exFDLli0bAQBjx45FMkm3f+kyhlPqxzN9\nlk5xDidPm4J8N+chNoYo/qarbgcAfP4GdqGPf5HXw2Y7hdnHkoK47i9MgPvicVTliHbjx16Mxkp6\nN0anqA4uYVLv4585BwAwsb4OTc2KesrrUXVr6urk+cpLR63aqjTyEqYo5oi204MH6CWEHUmxFqGW\ntT9vJ43yzEn0Bk5fzeTvA4mf4gcf4v6/+LCEXIpKQZGbUZetYZhQkkXnHUqv5Z5lcv4VlRYhfIfX\nyMFTOJ4Xm7jd209jEv/RRx7DwZvo/ZhCad0k7bYqxHMsl/UoBDu1cR9DC89phK5NmzZtw8TeeYQe\nGR+JI33R7BaFs27LASawOMjYwGKhtjbGB2fOnNlnC2EY7pWIr96fNGnSgFhy2Ls8F8CRL7yIFpVg\nE4Ty0ikUZDL6q9D9k+Yr9R4F9FBSKkyliO6U7rgr8e2sYyElTUDrbr1ZxiUl33Iszc3N0TZUErS7\nu2/streUgvrOKOmK09jYqL4FAKgdUQMU+HszThrfqBsoF+DdeDX3XciXtqe6GTn7QFtUcvBKiCtj\nYv1dlHKo/zeKou2qJ3XQz0rnehjwTOmvKhr7qmN9TzWTbUYNaWt2GCIUNURbPBpvFuPLI2fR+wtM\nL4qHFyQOPvVzV/QZZ2ACnhQbhd1E5p2L/gYASBeJRMfXSS4HAFTnJGffqtO6u3hvzJvHsS5btgod\nBIAYNYrHOHkyvYiqSh5fOlkGX5KwxR38/fyTqUZ56KOcj8YCk5lLrGOx491UryyupIpk5dqlAIA1\nE+klZ084C2YrrzUnSa+g63nmHz7+FXp3Dz+7AiOzlMhIJuUaFmH0WIxJZDPJseQLnbCkqE6JXw3F\nlDyESuK7hhNJPoxN8zpdvJWJ9ENG0Ls7c/FF+CmoJvo9QdCX/U7kLKKEp6wRpgdbztcdS5QCLD87\n9zccbypuAx69Fi/OxPUPzuB27n+I83fYlj+gRwTOWvMc3yNruO85Vbyv5zQwmds77xQaQ7tWNELX\npk2btmFi+w9CF2b+3+YxHj51EZ9sCc/GjHspCrVi3qH8rttXFGoo9lYovnoB5Vfb4mmYgngtkRG1\ndjdxmEModBqMWYLQS1IDYST3qcaZTPE0ZaQ7SzoGbIqz446K+6tRNV3OfECVoNLW1tYIffcv8rn4\n4osBAHfccUf03ubNLOtXRUNbpZR89OjRaBThrgm3UYZXcbUKUt6eSKsyei8qpbb8oWXpAcAVfW5f\nKFvHvPtobBCPYd0viNSrTmdcPD+N1Lk8HMRD1Veyr+2pi9UAKWYxSx1UEKLUlnUvpfpmEVmRWV3z\nOzI8kms3AgAWfJIa+7Es8xaB70X73LBhAwBg+rSD9zYFezRV9NTWyjnNllUhnpA4bKXEWkWuoKWF\nXmy6PoaRo/m7N1YQUZfVsKDo/EtISf3UKR8HAFTeOQkdzdxOzdmMoTfdwg7148+5BADQiA5khaa4\n8wFKStxxM3MrYyfSu0vAQdN2eogtHdJ1q4rzlJAciG0T0fZ09qAnJwJp6SG0KhILoxtH5XEMQGQF\nDqmid+AJG0WdzjBwMXvp1wEAC/MnAQBuO/sYAMBFv5RvBcp7N+FHRUclOWcAUM3LuopFWJKTuf3D\nZM888RiZeLPXUj7YRBppoeG6Zfz9KQdyfFWJvgV6YRhEBZX+EJcbjdC1adOmbZjY/oPQ5dliCMd3\n1GKWbzceengEkKY//woAYPm7yGSx5LvqwTlYIRuFlFQ/zrITGDesvu56taFIgmDJ/Ll8L9iH7vWD\nMMUj7zW6SHxfIXP1nURKyW8GyH3u8wBKJzAvgee2DilwSREBpVIpZDKMM6u4uEIsI0eyc9HFF1+M\n228n86GsjAhDlf5XVlZG/0el/lKUY0tnmcrvkCXh3nQDACDw/ZL3tA86VIrTq8ryWztacfSc93Bf\nwtT5+0NP8vi2EQkm3ntyxJO3BzBrIgrEP9y32esaKjEnJMeihJzk+qnwu/DKtylBO04YHxdeyzmw\nJKYKV2L0QYl1VVc39h+OY0+WyxN5Ll/OY66uySKW4BWwYT3fmzufRT/V0pGpo6MLGzcQmY8ZQ3Tc\n4fH81XcQuZ98rtR3vPwCNh3EBh95YbL4p3wSANCVJufc9rKAzbqQuWU8FzPHkRe/ehHrOyaMnYbd\nUiefF2G5dAPHE8g5KCpmSzHA9m2Mcbe0bgQAnHhSqZ7kH1npnhfUHHrISNco9ZnVT4nLCIFyj4yV\nzLJfAwD+vOY5AMDtF1Cy4DO/FwmHYrLk2QoKV1K5geSMQsPHT8/nnC58gtuZvPo+GQPH5YXFaDtK\natlzJL8QqmtEmtXAiI5nqOJc+82CruhfavxPSwXg7EWLURBtaVdGO+MlFjJ4/8WiknV3UrMhCC2o\n5KpaUEw5iaGqkgxt+BZvrvcupObDlkRaDYIvho/ma1k8Y0rnorc3FVqyZDLZ5//eYZFMou/pyQud\nb2FrPcZJkYsvtKauq5iQUuGV9vb2aHsFaU6sFvDLv0y6WkKKPurr66OG2VXVpLapYqTOLj4gaqpr\n0NpKXY7uG+lql1/BOVJa8QlpJRZ0O1HxRX/66GDMsaUptlDJtr9ponMrGxr70t1mYuWxAIC1L7BV\nV8e2bZh49kcBADkw3GTKHBlSnBGFy0J7oJym0oqJFCPNEjVVQj85KXyZ2MGxPPP976I+R6rkmZ/k\nA7ab+UXY8l1TGgvHrRSWLSJIMQw+YN9z2hGDnhOg9JCNxSXJnWtBbS0JAylJPj79FAvCKmRBnzRp\nNA47jElQS0JjHRmZjwYu8HdeyaR23ayjMHaK0HvjfDCMnM3FPkjzOCzXQvFqhmE+ey91g9aspW7M\nVOmo5KQcbG1k6K5yFHXPW1pEVVQWLUeqUzPpcsTiDA/NmDn0XgNRtbcx8L2o7WT/ynA/QCgAaEkL\nHybpGDWLnvgar6FvfZXH9t2HdypGKnqKEoaVymFXqoF/9skcHn2Ia8nU1Q/w+KSQyu+1IPt+r45o\nAKpiopTqK7CoQoahLOpabVGbNm3a/mVtv0Ho/U1a82Hx3Lk44jk25+0QvY2iZKucTzPxNOWcTwMA\nxuVzeOrcTwEAgk10CwMp0ik/iS77zCuuRLsoyG2CQu0Kf0vnnZ//FM0P8klberq/PYVE/a0s5ch+\n+ESOOwYmZYliYpIENQQJNAnrsOrb10So2JQEqmMTbdXVk4q2exeRmhlLorWNKCQuiHVMLZNXu3YT\nTnbnYqipY+n01ka67iMaiJY6dpNaNe6Iw1FZS4RfeyA/2yzHoObIv4hJU/P7N8ARJGYl9kW3hL/p\nEK/XLdjYrbRATB5neRXR4OE1DAm98vr9WC+e2gEXsJCqKSU0RS9yvfhqhKWCIDFDboXeiCgqBwfn\nYOw20maX/OgXAICJ9nTMmMVOPEufIV3NshOyK0FdQr2MxTzEHdLoXGMf4lAAZh40GQCQy0nv1qAN\n23etBQCMq28AAOzcxblLpXiu/vzUc5g8WXSJROHwpDl8dcbTk9m2m8j/tm9djm889jsAQPIEItV0\ngfvckiRCz9z/fZxz7gUAgLI0Qy1F0dfZIY2aAzOGTDn36UsZfytBONqaRZKgkgnQpUvfQDotaFaO\ns76+fNBzosJzYdBP6x8DSRNKuTARhGiXa6G9k2PeKV7ckWM43qZ7uKYccdBN2NHEc7tMqS6G9FQf\nOJvn9paH3sRvr/oSAOC3nzpCxtVPbRG91D4lhKeSoEakJSU/+SfKFjVC16ZNm7ZhYvstQvcFNfth\nAYvmk8p40i9JX9w4jbSrSCNb4mGbUhlM+i2/o8j/ApAi5NXa+3eqX6jEnd94Nzu3mK1Nkfrh/1bs\nXFkiJn0UHcbXxmdDVEsS1IoRudgyiNUhCw+q4URKiVJDg/JvfIN/iKDVOFUybLjRd1W/TNUVqka8\nFxhuJPY10VDUKX40VSVwDBeebCcqT4oQkGhny2w1727GiBFKQ3zol9gVvyQarBvFeHnDuHKMqCGa\nVAVTxZCoubmbrzOPOBavvsainjXfuw0AcMinzwAA5LIcSyJJ9JU3PXQViHKdOHMYKnQe+qpjvMfe\nowB6nmWHorVPvgoAmD6ZFLe6MfPQ1CR01gTn1qjmdquEntfVzPeTsThMYXAaxX2jvu7azlh1fR1L\n2b1CGSBMv1SS406KumNzK1FleUUNps/k/bJtI2P/S1ZzzLFqnsnWJh78Mce/B9Z3WSxWf+KHAQBb\ns/zOxB1E1nZnC+aecDQAYMUL9FjGjucgDqhj3L2rcxs8pTkntNXGZibkR9ZO4rE0MTczorYBry9Z\nDgCYc+T4Ic9JfxTe+3/Vj7NHciCvbeY+v/fMKvzs7PkAgFcbeS7mHchr7asPMoFcEC9x2ksfwbSz\nqU553Qc4t1PLeM38+lGO+zfXXIpQ4upffXYjAOD6uQ0AAEuddJglD1E5jP1XF/V+uIfPBmkaoWvT\npk3bMLH9FqFHFhhRnOzxj57Ft+Rp+N7nKEe5MyUSokEAQ+kHmwpp9qUshYYVyV7WPEdxr6e/cGH/\nXf6fWV0Zn+AZaVZa5ZRKl6IOQOJRHHDV1wAAbb030F8Ct7+YT+hE34kEQaWje2ATRnmGHV0IURxP\nPBQj6rziDEBDUX5BRLAcwQc1Hz4b9tNENUWlOjYEE0Yb1u18EwCwdsdKhKo7jHhTpXHRyyi6eWRS\nRE5xYQNt+E9S0twiPR05bHihHXlnSu7Bl5h/Qr5kWn5USGK4RPY9gsyWbqds85s7VqEoDUcdocoF\nO4TVo+KjrjBtQjcSVMsKpfRkHDKkeRk9mvmClBQTFRDCE5ZGvpv772wVQTIpNJozexaeW0i67+SJ\n3F9ZBcdRnuWxNozld5ctW4af/5D01Q9dfg0AYPq5ZO8s+QlL5R+/8zuoHkfmSrnIT/igR1VGRwg7\n1jdj+evMsEwSYbRyGU9jE9kkCDgHaza8jnnzWSRXUa56OA3egwn6Vd6EJuDK/a3ov69t5x3zyDJ6\nKO+bWYfr/0hva4zI7766iuh9dytffYO/ff61Zjy7iOPDLyit/FSe2/vz7cIMy3qIS6FUfRmvxxd3\n0COZUaWKpVw40tTWVf1C5dZN9qMuB2EYXZfmEHm/GqFr06ZN2zCx/R+h94olKaRuSZHP0/PYyEBx\nzINkGhM++jEAwCGnkD+bF0Gpv99FznrnwudhCzfaw16Khd7e6v63NFEVRWVcKB2h06tLjEK3nIM1\n32ZseOw3v4pEjkgiFKGypvXsxtK1ixKrbdIRxugqwBJpzyphzXQLtz6Mcz9V8SxaZXuxDJFee6t0\nJ5ICkWx5Ci27KUealkIlW2KCXQXpbC9o4rBJ1cLCBeywnwcxCCuX/pxhoDqv+0DYVw4h6kWrEHYs\njVDQuy9dhHoM6VQkJfOqaChu+Dj6WErjtrfxuJcsZjw4FA8vtDw2SwBQtHiSEtK0wTKU3EIMpsyh\nEbFa+GKIx6UaxIdBAqHKYVhqdoZmyYRIQUiP2ZEjKjB2POsGnn+ByHdULdHzzl28LpYsWQ8v5DG+\n+NrTAIA5x7DT14ETGzge8WZj1nK45WSAXfZuFtTdeAcLp678PF/H1I3GTmEcbZEORfVjOYYtm5nP\nGF3bgITJv9csJ1KdcDDHfMCBjJO/sojjzWYdPP0MWWwfPXvocWNPVX8JA8UIDJhy3ygPa3qNNBuZ\nxQYhK3Y2Y3w94/1vrNkIAKgeyRxNWTmvmeoskXUylkRlltv+wCheI3lx754WzzQwTFSIe2IZ0kxG\n1qR4rCQjre6lEurmdlqkGZfyeGMmkIh0JzRC16ZNm7Z/Sft/gNAHWq9GZ/Iir93t2PC6j8wUAAAF\nyElEQVQTCk2p1/5mGEbE4f6/ROJ7s6qYVODJIZhWIZIRLcWs+dydl2EMz7ztfMAQFC/fHdVvu55q\n0JFwEe/Xl9DL8rPpp7NMPZEyseTXl/fdQO3AsebGSSxZZGXVPIZB3/6tYdd2xAMlQzp0NOo4qomI\nkjCNRXUJQdivF6QAYzMISqfTopdhO8J08lXVMOObRuii2M3tqF6iZiIjn8l+4EFC74gL6gpUT1E5\nXsP0YIi4mqqsNYUlpFhDCnUlUulSqzFn6NWzAJCS87araYuMsQppqfqsqSQSfnM9/y+65JZv3boD\nC5//OQAglPzKo3/iNReEnLys9LT0PA9uXtg/IpR1xvz3AQBWvfYsAODGl55EIPLJdoJoNvU648fl\nGUoBJNMuUhmi4fEH0Nsqr+R8rFzFKu/DpF2kY3XAL26R42I5fmVF3aDnJBK0kv/t0EBReW3CtLn+\neW5/pngSFgLUybw1vIusm5Y2Hvepk3jtVFfwGHe0FZEuJ2OpIOuNx6nF7BnSDrFtNyZUci5GSw/V\nMkHjf9tIttGx46oR79/fWJB+eUxVtiouvREtaX44NK/l/+WC/s/YUEtp/7etsYsnrLPAcU2psqTr\nykB1RGVBCJhKR93otzj0exjEDESRGxX8aLuQyoDPRGutidbyBQCAyR2P8p09TFNSAik9kjBSN0xR\nQiMpoWBm4AMWr/qCOfTCopGjWbjk+9xf3LbgiCJdsdi3TFrxDS3TjmQGXHmvKKXaZpQXlz6sZhj9\nPJPmQj56DKmAtnSNKXp5GEJdcwxZ9EWSIJCGkZ5RjDrcqPsuLCrdb3koyXmwzGSkax9P7JtjrHp7\n19XVyT59qCayiaQURllc2C0BCh1ta/DyYtI5fVVgpXRPouur9H9o8sHUJWGmStGi6YhxP+lcDxIW\nz0G3XIO5bv6fjFfKQJtw3tm3y7apW19bzQfE449ygduyfhEAYN67DkQ6xYW2LLMPaotB316gvuni\nzRaOxxXt9Qtmc+Fd3cIL9rCD65CVizcvhU/WCCEB9K39QbYqjTYJUaZSDMv0eJzjJ14iwLrplEnY\nXVBFQtyuku04aix/E7huSXNKrjGlRaW014tqbfLD6OHv65CLNm3atP1r2r8cQt/fbFOnSkzy/5xn\nISl0q6DEMwQAmOL6w/BK4Zh+XV48oy8yhAWEgkaMy4i+IW61LQVZruWi4sP/BgB4+RbqOM9NUzpA\ndR/vvc10qFx2Cb2EffdZNDxkBE0m9pZ4fgvLt22V45TwhRWgu13eUjrm0aCkqMkwS9+HOnRxvXPK\nixG6n2MjcBkmiEuCt7t5i9ogAMA2QgQ+5zYf1YP0T1YXS56S2peMp0cSjY6SaDATUQFczt23wu5c\nngn+MCZyEQhgOUoPnaGCOfPSchQ857/61eMwAunO5PBcWNKVR4XKlHhbaDLUBABxoSSWia56YQS3\n/9kLvoBTTyXhoCihI5UIfP+JVGo0LAfNzUygp9I85q1b+f/x7z4VAPDSi6R+vvbyZmRiDNW81MVE\n/mlnTB/0nBRU7wC5Ip7fXcAom+d0YpY3VVcXjyEu5zbh+tjcyqRtbbWcP3FffaNvsh1GAS/u4thn\nWpwnX4TFshWck3zOR2VShR2lEK/YN4ziBSFCpfroCvqWa0fp13XkOedLG3ejTsTVRibjg54LQCN0\nbdq0aRs2ZuxvMWVt2rRp07ZvphG6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6\nNm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3a\ntA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T\n0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T0wu6Nm3atA0T+x/f\nMBCxbq8ovwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f3b4e28bba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### Load the images and plot them here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "new_images = np.zeros([10, 32, 32, 3])\n",
    "#list containing images for loading\n",
    "f_list = ['new_images/30km.jpg', 'new_images/child.jpeg', 'new_images/slippery.jpg','new_images/img2.jpg','new_images/stop.jpg','new_images/img3.jpg','new_images/rightoff.jpg','new_images/img1.jpg','new_images/img4.jpg','new_images/img5.jpg']\n",
    "print(len(f_list))\n",
    "\n",
    "for i, t in enumerate(f_list):\n",
    "    test_image = cv2.imread(t)\n",
    "    new_image = cv2.cvtColor(test_image, cv2.COLOR_BGR2RGB)\n",
    "    new_images[i] = cv2.resize(new_image, (32, 32))\n",
    "    \n",
    "    plt.subplot(2, 5, i+1)\n",
    "    plt.imshow(new_images[i])\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Correct image labels\n",
    "new_images_label = [1, 28, 23, 17, 14, 2, 11, 14, 13, 34]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict the Sign Type for Each Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32, 32, 1)\n",
      "[ 1 28 24 17 14  2 11 14 13 12]\n"
     ]
    }
   ],
   "source": [
    "### Run the predictions here and use the model to output the prediction for each image.\n",
    "\n",
    "test_image_num = len(new_images)\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, tf.train.latest_checkpoint('.')) #restoring the previously saved model\n",
    "    \n",
    "    new_images_normalized = gray_normalize(new_images)\n",
    "    results = np.argmax(np.array(sess.run(logits, feed_dict={x: new_images_normalized,keep_prob:0.5})),axis = 1)\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y  vs y_hat\n",
      "01:  01\n",
      "28:  28\n",
      "23:  24\n",
      "17:  17\n",
      "14:  14\n",
      "02:  02\n",
      "11:  11\n",
      "14:  14\n",
      "13:  13\n",
      "34:  12\n",
      "Accuracy: 0.80\n"
     ]
    }
   ],
   "source": [
    "### Calculate the accuracy for these 5 new images. \n",
    "### For example, if the model predicted 1 out of 5 signs correctly, it's 20% accurate on these new images.\n",
    "print(\"y  vs y_hat\")\n",
    "for i, result in enumerate(results):\n",
    "    print('{:02d}:  {:02d}'.format(new_images_label[i], result))\n",
    "print('Accuracy: {:.02f}'.format(np.sum(new_images_label==results)/len(new_images_label))) #accuracy on these new images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output Top 5 Softmax Probabilities For Each Image Found on the Web"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each of the new images, print out the model's softmax probabilities to show the **certainty** of the model's predictions (limit the output to the top 5 probabilities for each image). [`tf.nn.top_k`](https://www.tensorflow.org/versions/r0.12/api_docs/python/nn.html#top_k) could prove helpful here. \n",
    "\n",
    "The example below demonstrates how tf.nn.top_k can be used to find the top k predictions for each image.\n",
    "\n",
    "`tf.nn.top_k` will return the values and indices (class ids) of the top k predictions. So if k=3, for each sign, it'll return the 3 largest probabilities (out of a possible 43) and the correspoding class ids.\n",
    "\n",
    "Take this numpy array as an example. The values in the array represent predictions. The array contains softmax probabilities for five candidate images with six possible classes. `tf.nn.top_k` is used to choose the three classes with the highest probability:\n",
    "\n",
    "```\n",
    "# (5, 6) array\n",
    "a = np.array([[ 0.24879643,  0.07032244,  0.12641572,  0.34763842,  0.07893497,\n",
    "         0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.08594638,  0.0178669 ,  0.18063401,\n",
    "         0.15899337],\n",
    "       [ 0.26076848,  0.23664738,  0.08020603,  0.07001922,  0.1134371 ,\n",
    "         0.23892179],\n",
    "       [ 0.11943333,  0.29198961,  0.02605103,  0.26234032,  0.1351348 ,\n",
    "         0.16505091],\n",
    "       [ 0.09561176,  0.34396535,  0.0643941 ,  0.16240774,  0.24206137,\n",
    "         0.09155967]])\n",
    "```\n",
    "\n",
    "Running it through `sess.run(tf.nn.top_k(tf.constant(a), k=3))` produces:\n",
    "\n",
    "```\n",
    "TopKV2(values=array([[ 0.34763842,  0.24879643,  0.12789202],\n",
    "       [ 0.28086119,  0.27569815,  0.18063401],\n",
    "       [ 0.26076848,  0.23892179,  0.23664738],\n",
    "       [ 0.29198961,  0.26234032,  0.16505091],\n",
    "       [ 0.34396535,  0.24206137,  0.16240774]]), indices=array([[3, 0, 5],\n",
    "       [0, 1, 4],\n",
    "       [0, 5, 1],\n",
    "       [1, 3, 5],\n",
    "       [1, 4, 3]], dtype=int32))\n",
    "```\n",
    "\n",
    "Looking just at the first row we get `[ 0.34763842,  0.24879643,  0.12789202]`, you can confirm these are the 3 largest probabilities in `a`. You'll also notice `[3, 0, 5]` are the corresponding indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 5 Predictions for each image :\n",
      " [[ 1  0 23 21 22]\n",
      " [28 27 24 29 20]\n",
      " [24 19 23  0 37]\n",
      " [17 16 34 40 38]\n",
      " [14 13 34 38  9]\n",
      " [ 2  1  0  3  4]\n",
      " [11 21 30 27  0]\n",
      " [14  3 34  2 38]\n",
      " [13 42  9 15 34]\n",
      " [12 11 34 23  0]]\n",
      "Top 5 corresponding probabilities:\n",
      " [[  9.88838971e-01   1.11609772e-02   7.37002690e-13   4.81171960e-13\n",
      "    3.10117383e-13]\n",
      " [  1.00000000e+00   3.01840996e-12   6.04277409e-21   1.06777207e-23\n",
      "    2.38694149e-25]\n",
      " [  9.99839544e-01   1.60463474e-04   1.77576519e-12   2.84468114e-13\n",
      "    5.80769767e-14]\n",
      " [  9.99791682e-01   2.02283045e-04   3.06682273e-06   3.02087733e-06\n",
      "    1.37396251e-19]\n",
      " [  1.00000000e+00   3.11388493e-09   8.77106073e-14   5.27649417e-14\n",
      "    1.53828840e-14]\n",
      " [  1.00000000e+00   2.94529137e-30   0.00000000e+00   0.00000000e+00\n",
      "    0.00000000e+00]\n",
      " [  1.00000000e+00   7.29954893e-29   6.25719856e-30   1.96136746e-36\n",
      "    0.00000000e+00]\n",
      " [  1.00000000e+00   6.35365760e-09   4.38445417e-12   4.15141986e-15\n",
      "    1.57578386e-17]\n",
      " [  1.00000000e+00   3.16477533e-27   1.67502999e-30   1.00211881e-31\n",
      "    9.80466908e-32]\n",
      " [  1.00000000e+00   2.20627943e-11   8.85438302e-14   1.15720932e-15\n",
      "    2.03396500e-16]]\n"
     ]
    }
   ],
   "source": [
    "### Print out the top five softmax probabilities for the predictions on the German traffic sign images found on the web. \n",
    "### Feel free to use as many code cells as needed.\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, tf.train.latest_checkpoint('.'))\n",
    "    softmax = sess.run(tf.nn.softmax(logits), feed_dict = {x:new_images_normalized,keep_prob:0.5})\n",
    "    top5 = sess.run(tf.nn.top_k(softmax, k=5))\n",
    "\n",
    "    #print(top_5)\n",
    "    print(\"Top 5 Predictions for each image :\\n\", top5.indices)\n",
    "    print(\"Top 5 corresponding probabilities:\\n\", top5.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Writeup\n",
    "\n",
    "Once you have completed the code implementation, document your results in a project writeup using this [template](https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project/blob/master/writeup_template.md) as a guide. The writeup can be in a markdown or pdf file. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Note**: Once you have completed all of the code implementations and successfully answered each question above, you may finalize your work by exporting the iPython Notebook as an HTML document. You can do this by using the menu above and navigating to  \\n\",\n",
    "    \"**File -> Download as -> HTML (.html)**. Include the finished document along with this notebook as your submission."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Step 4 (Optional): Visualize the Neural Network's State with Test Images\n",
    "\n",
    " This Section is not required to complete but acts as an additional excersise for understaning the output of a neural network's weights. While neural networks can be a great learning device they are often referred to as a black box. We can understand what the weights of a neural network look like better by plotting their feature maps. After successfully training your neural network you can see what it's feature maps look like by plotting the output of the network's weight layers in response to a test stimuli image. From these plotted feature maps, it's possible to see what characteristics of an image the network finds interesting. For a sign, maybe the inner network feature maps react with high activation to the sign's boundary outline or to the contrast in the sign's painted symbol.\n",
    "\n",
    " Provided for you below is the function code that allows you to get the visualization output of any tensorflow weight layer you want. The inputs to the function should be a stimuli image, one used during training or a new one you provided, and then the tensorflow variable name that represents the layer's state during the training process, for instance if you wanted to see what the [LeNet lab's](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) feature maps looked like for it's second convolutional layer you could enter conv2 as the tf_activation variable.\n",
    "\n",
    "For an example of what feature map outputs look like, check out NVIDIA's results in their paper [End-to-End Deep Learning for Self-Driving Cars](https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/) in the section Visualization of internal CNN State. NVIDIA was able to show that their network's inner weights had high activations to road boundary lines by comparing feature maps from an image with a clear path to one without. Try experimenting with a similar test to show that your trained network's weights are looking for interesting features, whether it's looking at differences in feature maps from images with or without a sign, or even what feature maps look like in a trained network vs a completely untrained one on the same sign image.\n",
    "\n",
    "<figure>\n",
    " <img src=\"visualize_cnn.png\" width=\"380\" alt=\"Combined Image\" />\n",
    " <figcaption>\n",
    " <p></p> \n",
    " <p style=\"text-align: center;\"> Your output should look something like this (above)</p> \n",
    " </figcaption>\n",
    "</figure>\n",
    " <p></p> \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Visualize your network's feature maps here.\n",
    "### Feel free to use as many code cells as needed.\n",
    "\n",
    "# image_input: the test image being fed into the network to produce the feature maps\n",
    "# tf_activation: should be a tf variable name used during your training procedure that represents the calculated state of a specific weight layer\n",
    "# activation_min/max: can be used to view the activation contrast in more detail, by default matplot sets min and max to the actual min and max values of the output\n",
    "# plt_num: used to plot out multiple different weight feature map sets on the same block, just extend the plt number for each new feature map entry\n",
    "\n",
    "def outputFeatureMap(image_input, tf_activation, activation_min=-1, activation_max=-1 ,plt_num=1):\n",
    "    # Here make sure to preprocess your image_input in a way your network expects\n",
    "    # with size, normalization, ect if needed\n",
    "    # image_input =\n",
    "    # Note: x should be the same name as your network's tensorflow data placeholder variable\n",
    "    # If you get an error tf_activation is not defined it may be having trouble accessing the variable from inside a function\n",
    "    activation = tf_activation.eval(session=sess,feed_dict={x : image_input})\n",
    "    featuremaps = activation.shape[3]\n",
    "    plt.figure(plt_num, figsize=(15,15))\n",
    "    for featuremap in range(featuremaps):\n",
    "        plt.subplot(6,8, featuremap+1) # sets the number of feature maps to show on each row and column\n",
    "        plt.title('FeatureMap ' + str(featuremap)) # displays the feature map number\n",
    "        if activation_min != -1 & activation_max != -1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmin =activation_min, vmax=activation_max, cmap=\"gray\")\n",
    "        elif activation_max != -1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmax=activation_max, cmap=\"gray\")\n",
    "        elif activation_min !=-1:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", vmin=activation_min, cmap=\"gray\")\n",
    "        else:\n",
    "            plt.imshow(activation[0,:,:, featuremap], interpolation=\"nearest\", cmap=\"gray\")"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}