{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Model evaluation & cross-validation\n", "\n", "````{margin}\n", "```{warning}\n", "These pages are currently under construction and will be updated continuously.\n", "Please visit these pages again in the next few weeks for further information.\n", "````\n", "\n", "---------------" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Aim(s) of this section 🎯\n", "\n", "As mention in the previous section, it is not sufficient to apply these methods to learn somthing about the nature of our data. It is always necessary to assess the quality of the implemented model. The goal of these section is to look at ways to estimate the generalization accuracy of a model on future (e.g.,unseen, out-of-sample) data.\n", "\n", "In other words, at the end of these sections you should know:\n", "- 1) different techniques to evaluate a given model\n", "- 2) understand the basic idea of cross-validation and different kinds of the same\n", "- 3) get an idea how to assess the significance (e.g., via permutation tests)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Outline for this section 📝\n", "\n", "1. Model diagnostics\n", "\n", "2. Cross-validation" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Prepare data for model\n", "\n", "Lets bring back our example data set (you know the song ...)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "There are 155 samples and 2016 features\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "\n", "# get the data set\n", "data = np.load('MAIN2019_BASC064_subsamp_features.npz')['a']\n", "\n", "# get the labels\n", "info = pd.read_csv('participants.csv')\n", "\n", "\n", "print('There are %s samples and %s features' % (data.shape[0], data.shape[1]))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now let's look at the labels" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": false, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
participant_idAgeAgeGroupChild_AdultGenderHandedness
0sub-pixar12327.06AdultadultFR
1sub-pixar12433.44AdultadultMR
2sub-pixar12531.00AdultadultMR
3sub-pixar12619.00AdultadultFR
4sub-pixar12723.00AdultadultFR
\n", "
" ], "text/plain": [ " participant_id Age AgeGroup Child_Adult Gender Handedness\n", "0 sub-pixar123 27.06 Adult adult F R\n", "1 sub-pixar124 33.44 Adult adult M R\n", "2 sub-pixar125 31.00 Adult adult M R\n", "3 sub-pixar126 19.00 Adult adult F R\n", "4 sub-pixar127 23.00 Adult adult F R" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "info.head(n=5)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "We'll set `Age` as target\n", "- i.e., well look at these from the `regression` perspective" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": true, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "count 155.000000\n", "mean 10.555189\n", "std 8.071957\n", "min 3.518138\n", "25% 5.300000\n", "50% 7.680000\n", "75% 10.975000\n", "max 39.000000\n", "Name: Age, dtype: float64" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# set age as target\n", "Y_con = info['Age']\n", "Y_con.describe()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Next:\n", "- we need to divide our input data `X` into `training` and `test` sets" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "# import necessary python modules\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.svm import SVC\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import accuracy_score\n", "\n", "# split the data\n", "X_train, X_test, y_train, y_test = train_test_split(data, Y_con, random_state=0)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now lets look at the size of the data sets" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "N used for training: 116 | N used for testing: 39\n" ] } ], "source": [ "# print the size of our training and test groups\n", "print('N used for training:', len(X_train),\n", " ' | N used for testing:', len(X_test))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "**Question:** Is that a good distribution? Does it look ok?\n", "\n", "- Why might this be problematic (hint: what do you know about groups (e.g., `Child_Adult`) in the data." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAATsUlEQVR4nO3dfbBcBXmA8eclCUkEOgnhTgpJaOLHIBTbUCLlazoWayfaD6JNQcbadAYbHLCDY7Wi/aPa1hmsVgWnilSpsVUBRQpSxAIGrQNCLxjkKxJQMBeBhGgQOhMl8PaPPRcuMbm5CXv23Xv3+c3cye45+/HeM+Th5Ozu2chMJEm9t0/1AJI0qAywJBUxwJJUxABLUhEDLElFplcPMBHLly/Pa665pnoMSdpTMd7KSbEH/Nhjj1WPIEldNykCLElTkQGWpCIGWJKKTIoX4SRNTk899RQjIyNs27atepRWzZo1i4ULFzJjxow9up8BltSakZERDjjgABYvXkzEuG8ImLQyky1btjAyMsKSJUv26L4egpDUmm3btjFv3rwpG1+AiGDevHl7tZdvgCW1airHd9Te/o4GWJKKGGBJPbNg0aFERNd+Fiw6dNzn27p1K5/4xCf2eM7Xve51bN26dS9/y4nzRThJPfPjkY2c+qkbu/Z4l5xx/LjrRwN85plnPm/59u3bmT591/m7+uqruzLf7hhgSVPWOeecw/3338/SpUuZMWMGs2bNYu7cuaxfv557772XFStWsHHjRrZt28bZZ5/N6tWrAVi8eDHDw8M8+eSTvPa1r+XEE0/kxhtvZMGCBVxxxRXMnj27K/N5CELSlHXuuefykpe8hHXr1vGhD32I2267jfPOO497770XgIsuuohbb72V4eFhzj//fLZs2fJLj7FhwwbOOuss7rrrLubMmcNll13WtfncA5Y0MI455pjnvVf3/PPP5/LLLwdg48aNbNiwgXnz5j3vPkuWLGHp0qUAHH300TzwwANdm2dK7wF3+4D/RA76S+pf++2337OXb7jhBq677jpuuukmbr/9do466qidvpd35syZz16eNm0a27dv79o8U3oPuNsH/GH3B/0l9Y8DDjiAJ554YqfrHn/8cebOncuLXvQi1q9fz3e+850eTzfFAyypvxyycFFXd2IOWbho3PXz5s3jhBNO4Mgjj2T27NnMnz//2XXLly/nggsu4PDDD+ewww7j2GOP7dpcE2WAJfXMQxt/1PPn/MIXvrDT5TNnzuRrX/vaTteNHuc96KCDuPPOO59d/s53vrOrs03pY8CS1M8MsCQVMcCSWpWZ1SO0bm9/RwMsqTWzZs1iy5YtUzrCo+cDnjVr1h7f1xfhJLVm4cKFjIyMsHnz5upRWjX6jRh7ygBLas2MGTP2+FsiBomHICSpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkoq0HuCImBYR342Iq5rrSyLi5oi4LyIuiYh9255BkvpRL/aAzwbuGXP9g8BHM/OlwE+B03swgyT1nVYDHBELgT8APt1cD+Ak4MvNTdYAK9qcQZL6Vdt7wB8D/gZ4prk+D9iamdub6yPAgpZnkKS+1FqAI+IPgU2Zeete3n91RAxHxPDmzZu7PJ0k1WtzD/gE4I8j4gHgYjqHHs4D5kTE9OY2C4GHdnbnzLwwM5dl5rKhoaEWx5SkGq0FODPfk5kLM3Mx8EbgG5n5JmAtsLK52SrgirZmkKR+VvE+4HcD74iI++gcE/5MwQySVG767m/ywmXmDcANzeUfAMf04nklqZ/5SThJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSirQW4IiYFRG3RMTtEXFXRLy/Wb4kIm6OiPsi4pKI2LetGSSpn7W5B/xz4KTM/E1gKbA8Io4FPgh8NDNfCvwUOL3FGSSpb7UW4Ox4srk6o/lJ4CTgy83yNcCKtmaQpH7W6jHgiJgWEeuATcC1wP3A1szc3txkBFjQ5gyS1K9aDXBmPp2ZS4GFwDHAyyd634hYHRHDETG8efPmtkaUpDI9eRdEZm4F1gLHAXMiYnqzaiHw0C7uc2FmLsvMZUNDQ70YU5J6qs13QQxFxJzm8mzgNcA9dEK8srnZKuCKtmaQpH42ffc32WsHA2siYhqd0F+amVdFxN3AxRHxj8B3gc+0OIMk9a3WApyZ3wOO2snyH9A5HixJA81PwklSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklRkQgGOiBMmskySNHET3QP++ASXSZImaPp4KyPiOOB4YCgi3jFm1a8A09ocTJKmunEDDOwL7N/c7oAxy38GrGxrKEkaBOMGODO/CXwzIj6bmQ/2aCZJGgi72wMeNTMiLgQWj71PZp7UxlCSNAgmGuAvARcAnwaebm8cSRocEw3w9sz8ZKuTSNKAmejb0L4aEWdGxMERceDoT6uTSdIUN9E94FXNn+8asyyBF3d3HEkaHBMKcGYuaXsQSRo0EwpwRPz5zpZn5ue6O44kDY6JHoJ45ZjLs4BXA7cBBliS9tJED0H81djrETEHuLiNgSRpUOzt6Sj/D/C4sCS9ABM9BvxVOu96gM5JeA4HLm1rKEkaBBM9BvzhMZe3Aw9m5kgL80jSwJjQIYjmpDzr6ZwRbS7wizaHkqRBMNFvxDgFuAX4U+AU4OaI8HSUkvQCTPQQxN8Cr8zMTQARMQRcB3y5rcEkaaqb6Lsg9hmNb2PLHtxXkrQTE90DviYivg58sbl+KnB1OyNJ0mDY3XfCvRSYn5nviog3ACc2q24CPt/2cJI0le1uD/hjwHsAMvMrwFcAIuIVzbo/anE2SZrSdnccd35m3rHjwmbZ4lYmkqQBsbsAzxln3ewuziFJA2d3AR6OiL/ccWFEvAW4tZ2RJGkw7O4Y8NuByyPiTTwX3GXAvsDrW5xLkqa8cQOcmY8Cx0fE7wJHNov/KzO/0fpkkjTFTfR8wGuBtS3PMjnsM52I6OpDHrJwEQ9t/FFXH1NS/5voBzE06pntnPqpG7v6kJeccXxXH0/S5ODHiSWpSGsBjohFEbE2Iu6OiLsi4uxm+YERcW1EbGj+nNvWDJLUz9rcA94O/HVmHgEcC5wVEUcA5wDXZ+bLgOub65I0cFoLcGY+nJm3NZefAO4BFgAnA2uam60BVrQ1gyT1s54cA46IxcBRwM10Pt78cLPqEWB+L2aQpH7TeoAjYn/gMuDtmfmzsesyM3nuyz53vN/qiBiOiOHNmze3PaYk9VyrAY6IGXTi+/nmbGoAj0bEwc36g4FNO7tvZl6Ymcsyc9nQ0FCbY0pSiTbfBRHAZ4B7MvMjY1ZdCaxqLq8CrmhrBknqZ21+EOME4M3AHRGxrln2XuBc4NKIOB14kM6XfErSwGktwJn5bWBXn9l9dVvPK0mThZ+Ek6QiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAe4HzffMdfNnwaJDq38rSbvhd8L1A79nThpI7gFLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBVpLcARcVFEbIqIO8csOzAiro2IDc2fc9t6fknqd23uAX8WWL7DsnOA6zPzZcD1zXVJGkitBTgzvwX8ZIfFJwNrmstrgBVtPb8k9bteHwOen5kPN5cfAebv6oYRsToihiNiePPmzb2ZTpJ6qOxFuMxMIMdZf2FmLsvMZUNDQz2cTJJ6o9cBfjQiDgZo/tzU4+eXpL7R6wBfCaxqLq8Crujx80tS32jzbWhfBG4CDouIkYg4HTgXeE1EbAB+r7kuSQNpelsPnJmn7WLVq9t6TkmaTPwknCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcBT1T7TiYiu/ixYdGj1b1VqwaJD3abqqtZOR6liz2zn1E/d2NWHvOSM47v6eJPNj0c2uk3VVe4BS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLCkCfE78brP74STNCF+J173uQcsSUUMsCQVMcCSVMQAS1IRX4TTlLRg0aH8eGRj9Ri7t890IqKrDzltxkyefurnXX3M1rTw+x+ycBEPbfxRVx+zLQZYU9KkecX+me2tzNntxxx93K5r6fefLDwEIUlFDLAkFTHAklTEAEtSEQMsSUUMsKSppXlr22Q4aZBvQ5M0tUyit7a5ByxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSkZIAR8TyiPh+RNwXEedUzCBJ1Xoe4IiYBvwL8FrgCOC0iDii13NIUrWKPeBjgPsy8weZ+QvgYuDkgjkkqVRkZm+fMGIlsDwz39JcfzPw25n5th1utxpY3Vw9DPh+j0Y8CHisR8/VLc7cO5NxbmfujZ3N/FhmLt/VHfr2ZDyZeSFwYa+fNyKGM3NZr5/3hXDm3pmMcztzb+zNzBWHIB4CFo25vrBZJkkDpSLA/wu8LCKWRMS+wBuBKwvmkKRSPT8EkZnbI+JtwNeBacBFmXlXr+cYR88Pe3SBM/fOZJzbmXtjj2fu+YtwkqQOPwknSUUMsCQVMcBjRMQDEXFHRKyLiOHqeXYmIi6KiE0RceeYZQdGxLURsaH5c27ljDvaxczvi4iHmm29LiJeVznjjiJiUUSsjYi7I+KuiDi7Wd6323qcmft9W8+KiFsi4vZm7vc3y5dExM3NKQsuaV607wvjzPzZiPjhmG29dNzH8RjwcyLiAWBZZvbtG8Aj4neAJ4HPZeaRzbJ/An6Smec259aYm5nvrpxzrF3M/D7gycz8cOVsuxIRBwMHZ+ZtEXEAcCuwAvgL+nRbjzPzKfT3tg5gv8x8MiJmAN8GzgbeAXwlMy+OiAuA2zPzk5Wzjhpn5rcCV2XmlyfyOO4BTzKZ+S3gJzssPhlY01xeQ+cvXd/Yxcx9LTMfzszbmstPAPcAC+jjbT3OzH0tO55srs5ofhI4CRgNWb9t613NvEcM8PMl8N8RcWvzUejJYn5mPtxcfgSYXznMHnhbRHyvOUTRN/+U31FELAaOAm5mkmzrHWaGPt/WETEtItYBm4BrgfuBrZm5vbnJCH32P5MdZ87M0W39gWZbfzQiZo73GAb4+U7MzN+ic6a2s5p/Ok8q2TmmNBmOK30SeAmwFHgY+OfSaXYhIvYHLgPenpk/G7uuX7f1Tmbu+22dmU9n5lI6n4w9Bnh57US7t+PMEXEk8B46s78SOBAY9/CUAR4jMx9q/twEXE7nP4TJ4NHm+N/occBNxfPsVmY+2vwH/Azwr/Thtm6O7V0GfD4zv9Is7uttvbOZJ8O2HpWZW4G1wHHAnIgY/bBY356yYMzMy5vDQJmZPwf+jd1sawPciIj9mhcuiIj9gN8H7hz/Xn3jSmBVc3kVcEXhLBMyGrHG6+mzbd28yPIZ4J7M/MiYVX27rXc18yTY1kMRMae5PBt4DZ3j12uBlc3N+m1b72zm9WP+5xx0jlmPu619F0QjIl5MZ68XOh/R/kJmfqBwpJ2KiC8Cr6Jz6rtHgb8D/hO4FDgUeBA4JTP75kWvXcz8Kjr/JE7gAeCMMcdWy0XEicD/AHcAzzSL30vnmGpfbutxZj6N/t7Wv0HnRbZpdHYKL83Mv2/+Tl5M55/y3wX+rNmzLDfOzN8AhoAA1gFvHfNi3S8/jgGWpBoegpCkIgZYkooYYEkqYoAlqYgBlqQiBlhTXkSsiIiMiL7/dJUGiwHWIDiNztmqTqseRBrLAGtKa86LcCJwOp0vgCUi9omIT0TE+uacvldHxMpm3dER8c3mhExf3+FTZFJXGWBNdScD12TmvcCWiDgaeAOwGDgCeDOd8w6Mnkfh48DKzDwauAjou09Dauro+bciSz12GnBec/ni5vp04EvNyWkeiYi1zfrDgCOBazsf5WcanbOHSa0wwJqyIuJAOif1fkVEJJ2gJs+d8+OX7gLclZnH9WhEDTgPQWgqWwn8e2b+WmYuzsxFwA/pfDvHnzTHgufTOTEQwPeBoYh49pBERPx6xeAaDAZYU9lp/PLe7mXAr9L5hoW7gf8AbgMez8xf0In2ByPidjpnszq+Z9Nq4Hg2NA2kiNi/+ULFecAtwAmZ+Uj1XBosHgPWoLqqOaH2vsA/GF9VcA9Ykop4DFiSihhgSSpigCWpiAGWpCIGWJKK/D8WGBmJv4D0egAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVvUlEQVR4nO3df7RdZX3n8fc3JBgxGWKSCzW/JtgKowMt4KUj4LQFSo3WBdKhIIJDOzhhxh+jowuEUerorK5F13QwXe1IyWCKLSwEEVctWEyKqdiFhV5ikPDDEqvVy69cYFCKAxj4zh9np73e3iQ3h3P299x736+1zrrn7L3Pfr73yc0nT56793MiM5EktW9OdQGSNFsZwJJUxACWpCIGsCQVMYAlqcjc6gKmYs2aNXnLLbdUlyFJ3Yjd7ZgWI+DHH3+8ugRJ6rlpEcCSNBMZwJJUxACWpCLT4pdwkmaOH//4x4yOjvLss89Wl9JT8+fPZ8WKFcybN2/K7zGAJbVqdHSUhQsXsnr1aiJ2e4HAtJKZPPHEE4yOjnLIIYdM+X1OQUhq1bPPPsuSJUtmTPgCRARLlizZ51G9ASypdTMpfHfp5nsygCWpiAEsqdTylauIiJ49lq9ctdc2n3rqKT71qU91Ve+6dev40Y9+1NV7J/KXcJJKPTz6fc684vaene+684/b6zG7Avjd7373Pp9/3bp1nHPOORxwwAHdlPcT+hbAEbEBeCuwIzMPH7f9fcB7gBeAmzPzwn7VIEmTueiii/j2t7/NkUceycknn8xBBx3E9ddfz3PPPcdpp53Gxz/+cZ555hnOOOMMRkdHeeGFF7jkkkt47LHHePjhhznhhBNYunQpmzdvfkl19HMEfBXwB8Af79oQEScApwI/l5nPRcRBfWxfkiZ16aWXsm3bNrZu3crGjRu54YYbuPPOO8lMTjnlFG677TbGxsZYtmwZN998MwA/+MEPOPDAA7nsssvYvHkzS5cufcl19G0OODNvA56csPk/A5dm5nPNMTv61b4kTcXGjRvZuHEjRx11FEcffTQPPPAADz74IEcccQSbNm3iwx/+MF/72tc48MADe95227+EOxT4txFxR0R8NSKO2d2BEbE2IkYiYmRsbKyrxiom9yVNL5nJxRdfzNatW9m6dSvbt2/nvPPO49BDD2XLli0cccQRfPSjH+UTn/hEz9tu+5dwc4HFwBuAY4DrI+LVOclHM2fmemA9wPDwcFcf3VwxuS9p8C1cuJCnn34agDe96U1ccsklnH322SxYsICHHnqIefPmsXPnThYvXsw555zDokWLuPLKK3/ivb2Ygmg7gEeBG5vAvTMiXgSWAt0NcSVNe8tWrOzp4GbZipV7PWbJkiUcf/zxHH744bz5zW/mHe94B8ceeywACxYs4Oqrr2b79u1ccMEFzJkzh3nz5nH55ZcDsHbtWtasWcOyZcte8i/hYpLBZ89ExGrgpl1XQUTEfwKWZeZvRcShwK3AqslGwOMNDw/nyMhIN+33fATcz/6SZoP777+f1772tdVl9MVuvrfd3iLXz8vQrgV+CVgaEaPAx4ANwIaI2AY8D5y7t/CVpJmqbwGcmWftZtc5/WpTkqYTb0WW1LqZ+B/fbr4nA1hSq+bPn88TTzwxo0J413rA8+fP36f3uRaEpFatWLGC0dFRur2+f1Dt+kSMfWEAS2rVvHnz9ulTI2YypyAkqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCJ9C+CI2BAROyJi2yT7PhQRGRFL+9W+JA26fo6ArwLWTNwYESuBXwG+18e2JWng9S2AM/M24MlJdn0SuBDIfrUtSdNBq3PAEXEq8FBm3t1mu5I0iOa21VBEHAD8NzrTD1M5fi2wFmDVqlV9rEySarQ5Av5p4BDg7oj4LrAC2BIRPzXZwZm5PjOHM3N4aGioxTIlqR2tjYAz8x7goF2vmxAezszH26pBkgZJPy9Duxb4OnBYRIxGxHn9akuSpqO+jYAz86y97F/dr7YlaTrwTjhJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklSkbwEcERsiYkdEbBu37X9GxAMR8c2I+EJELOpX+5I06Po5Ar4KWDNh2ybg8Mz8WeBvgYv72L4kDbS+BXBm3gY8OWHbxszc2bz8a2BFv9qXpEFXOQf8H4A/393OiFgbESMRMTI2NtZiWZLUjpIAjoiPADuBa3Z3TGauz8zhzBweGhpqrzhJasncthuMiN8A3gqclJnZdvuSNChaDeCIWANcCPxiZv6ozbYladD08zK0a4GvA4dFxGhEnAf8AbAQ2BQRWyPiD/vVviQNur6NgDPzrEk2f7pf7UnSdOOdcJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqUjfAjgiNkTEjojYNm7b4ojYFBEPNl9f2a/2JWnQ9XMEfBWwZsK2i4BbM/M1wK3Na0malfoWwJl5G/DkhM2nAp9pnn8GeFu/2pekQdf2HPDBmflI8/xR4ODdHRgRayNiJCJGxsbG2qlOklpU9ku4zEwg97B/fWYOZ+bw0NBQi5VJUjvaDuDHIuJVAM3XHS23L0kDo+0A/iJwbvP8XOBPW25fkgZGPy9Duxb4OnBYRIxGxHnApcDJEfEg8MvNa0maleb268SZedZudp3UrzYlaTrxTjhJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKTCmAI+L4qWyTJE3dVEfAvz/FbTPbnLlERE8ey1euqv5uJBXb42poEXEscBwwFBEfHLfrXwD79bOwgfTiTs684vaenOq684/ryXkkTV97W45yf2BBc9zCcdt/CJzer6IkaTbYYwBn5leBr0bEVZn59y3VJEmzwlQXZH9ZRKwHVo9/T2ae2I+iJGk2mGoAfw74Q+BK4IX+lSNJs8dUA3hnZl7e10okaZaZ6mVofxYR746IV0XE4l2PvlYmSTPcVEfAuz5K/oJx2xJ4dW/LkaTZY0oBnJmH9LsQSZptphTAEfHvJ9uemX/c23IkafaY6hTEMeOezwdOArYABrAkdWmqUxDvG/86IhYBn+1HQZI0W3S7HOUzgPPCkvQSTHUO+M/oXPUAnUV4Xgtc36+iJGk2mOoc8O+Oe74T+PvMHO220Yj4r8C76IT6PcBvZuaz3Z5PkqajKU1BNIvyPEBnRbRXAs9322BELAf+CzCcmYfTGVG/vdvzSdJ0NdVPxDgDuBP4deAM4I6IeCnLUc4FXh4Rc4EDgIdfwrkkaVqa6hTER4BjMnMHQEQMAX8B3LCvDWbmQxHxu8D3gP8HbMzMjROPi4i1wFqAVav89AhJM89Ur4KYsyt8G0/sw3t/QkS8EjiVzlUUy4BXRMQ5E4/LzPWZOZyZw0NDQ900JUkDbaoj4Fsi4svAtc3rM4EvddnmLwPfycwxgIi4kc7HHl3d5fkkaVra22fC/QxwcGZeEBG/Bryx2fV14Jou2/we8IaIOIDOFMRJwEiX55KkaWtv0wjr6Hz+G5l5Y2Z+MDM/CHyh2bfPMvMOOnPHW+hcgjYHWN/NuSRpOtvbFMTBmXnPxI2ZeU9ErO620cz8GPCxbt8vSTPB3kbAi/aw7+U9rEOSZp29BfBIRPzHiRsj4l3AXf0pSZJmh71NQXwA+EJEnM0/Be4wsD9wWh/rkqQZb48BnJmPAcdFxAnA4c3mmzPzK32vTJJmuKmuB7wZ2NznWiRpVul2PWBJ0ktkAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJalISQBHxKKIuCEiHoiI+yPi2Io6JKnS3KJ2fw+4JTNPj4j9gQOK6pCkMq0HcEQcCPwC8BsAmfk88HzbdUhStYopiEOAMeCPIuIbEXFlRLxi4kERsTYiRiJiZGxsrP0qJanPKgJ4LnA0cHlmHgU8A1w08aDMXJ+Zw5k5PDQ01HaNktR3FQE8Coxm5h3N6xvoBLIkzSqtB3BmPgp8PyIOazadBNzXdh2SVK3qKoj3Adc0V0D8HfCbRXVIUpmSAM7MrcBwRduSNCi8E06SihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAq8yZS0T07LF85arq70jSPqpajlIv7uTMK27v2emuO/+4np1LUjscAUtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUpC+CI2C8ivhERN1XVIEmVKkfA7wfuL2xfkkqVBHBErAB+Fbiyon1JGgRVI+B1wIXAi0XtS1K51gM4It4K7MjMu/Zy3NqIGImIkbGxsZaqk6T2VIyAjwdOiYjvAp8FToyIqycelJnrM3M4M4eHhobarlGS+q71AM7MizNzRWauBt4OfCUzz2m7Dkmq5nXAklSk9FORM/Mvgb+srEGSqjgClqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAa9pZvnIVEdGzx/KVq6q/Jc1SpctRSt14ePT7nHnF7T0733XnH9ezc0n7whGwJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSkdYDOCJWRsTmiLgvIu6NiPe3XYMkDYKK5Sh3Ah/KzC0RsRC4KyI2ZeZ9BbVIUpnWR8CZ+UhmbmmePw3cDyxvuw5JqlY6BxwRq4GjgDsm2bc2IkYiYmRsbKz12iSp38oCOCIWAJ8HPpCZP5y4PzPXZ+ZwZg4PDQ21X6Ak9VlJAEfEPDrhe01m3lhRgyRVq7gKIoBPA/dn5mVtty9Jg6JiBHw88E7gxIjY2jzeUlCHJJVq/TK0zPwrINpuV5IGjXfCSVIRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAZ4o5c4mInj2Wr1zV0/KWr1zVs9p6rsd9N3f/+QP9Z9FLvfxzHfSfu37UV/GpyOqHF3dy5hW39+x0151/XM/OBfDw6Pd7Vl+va+tH3w3yn0Uv9fLPFQb75w56X58jYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIlARwRayLiWxGxPSIuqqhBkqq1HsARsR/wv4E3A68DzoqI17VdhyRVqxgB/zywPTP/LjOfBz4LnFpQhySVisxst8GI04E1mfmu5vU7gX+Tme+dcNxaYG3z8jDgW30ubSnweJ/b6Ja1dcfaumNt3dldbY9n5prJ3jCwn4iRmeuB9W21FxEjmTncVnv7wtq6Y23dsbbudFNbxRTEQ8DKca9XNNskaVapCOC/AV4TEYdExP7A24EvFtQhSaVan4LIzJ0R8V7gy8B+wIbMvLftOibR2nRHF6ytO9bWHWvrzj7X1vov4SRJHd4JJ0lFDGBJKmIAAxHx3Yi4JyK2RsRIcS0bImJHRGwbt21xRGyKiAebr68coNr+e0Q81PTd1oh4S1FtKyNic0TcFxH3RsT7m+3lfbeH2sr7LiLmR8SdEXF3U9vHm+2HRMQdzXIB1zW/MB+U2q6KiO+M67cj265tXI37RcQ3IuKm5vU+9ZsB/E9OyMwjB+Aaw6uAiRdtXwTcmpmvAW5tXle4in9eG8Anm747MjO/1HJNu+wEPpSZrwPeALynucV9EPpud7VBfd89B5yYmT8HHAmsiYg3AL/T1PYzwP8Fzhug2gAuGNdvWwtq2+X9wP3jXu9TvxnAAyYzbwOenLD5VOAzzfPPAG9rs6ZddlPbQMjMRzJzS/P8aTp/KZYzAH23h9rKZcc/NC/nNY8ETgRuaLZX9dvuahsIEbEC+FXgyuZ1sI/9ZgB3JLAxIu5qboEeNAdn5iPN80eBgyuLmcR7I+KbzRRFyfTIeBGxGjgKuIMB67sJtcEA9F3z3+itwA5gE/Bt4KnM3NkcMkrRPxgTa8vMXf32202/fTIiXlZRG7AOuBB4sXm9hH3sNwO4442ZeTSdFdreExG/UF3Q7mTnusGBGQUAlwM/Tee/iI8A/6uymIhYAHwe+EBm/nD8vuq+m6S2gei7zHwhM4+kc1fqzwP/qqKOyUysLSIOBy6mU+MxwGLgw23XFRFvBXZk5l0v5TwGMJCZDzVfdwBfoPNDOEgei4hXATRfdxTX848y87HmL8mLwP+hsO8iYh6dgLsmM29sNg9E301W2yD1XVPPU8Bm4FhgUUTsulGrfLmAcbWtaaZ0MjOfA/6Imn47HjglIr5LZ0XHE4HfYx/7bdYHcES8IiIW7noO/Aqwbc/vat0XgXOb5+cCf1pYy0/YFW6N0yjqu2b+7dPA/Zl52bhd5X23u9oGoe8iYigiFjXPXw6cTGeOejNwenNYVb9NVtsD4/5BDTpzrK33W2ZenJkrMnM1neUUvpKZZ7Ov/ZaZs/oBvBq4u3ncC3ykuJ5r6fx39Md05pDOozO3dCvwIPAXwOIBqu1PgHuAb9IJu1cV1fZGOtML3wS2No+3DELf7aG28r4Dfhb4RlPDNuC3mu2vBu4EtgOfA142QLV9pem3bcDVwIKKn7lxdf4ScFM3/eatyJJUZNZPQUhSFQNYkooYwJJUxACWpCIGsCQVMYA1K0TE2yIiI2Jg7vKSDGDNFmcBf9V8lQaCAawZr1mD4Y10bhx5e7NtTkR8KiIeaNYJ/lJEnN7se31EfLVZnOnLE+5Yk3rGANZscCpwS2b+LfBERLwe+DVgNfA64J101j/YtWbD7wOnZ+brgQ3Ab1cUrZmv9U9FlgqcRWehFOgsnHIWnZ/9z2VnIZxHI2Jzs/8w4HBgU2epAfajc/u11HMGsGa0iFhMZ6WqIyIi6QRq0ln1btK3APdm5rEtlahZzCkIzXSnA3+Smf8yM1dn5krgO3Q+2ePfNXPBB9NZUAXgW8BQRPzjlERE/OuKwjXzGcCa6c7in492Pw/8FJ0V3e6js6LWFuAHmfk8ndD+nYi4m87KZce1Vq1mFVdD06wVEQsy8x8iYgmdJQSPz8xHq+vS7OEcsGazm5oFv/cH/ofhq7Y5ApakIs4BS1IRA1iSihjAklTEAJakIgawJBX5/0R2OfbsYuj3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.displot(y_train,label='train')\n", "plt.legend()\n", "sns.displot(y_test,label='test')\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Model fit\n", "\n", "Now lets go ahead and fit the model\n", "- we will use a fairly standard regression model called a Support Vector Regressor (SVR)\n", " - similar to the one we used in the previous section" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "SVR(kernel='linear')" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.svm import SVR\n", "\n", "# define the model\n", "lin_svr = SVR(kernel='linear')\n", "\n", "# fit the model\n", "lin_svr.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Model diagnostics" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Now let's look at how the model performs in predicting the data\n", "- we can use the `score` method to calculate the coefficient of determination (or [R-squared](https://en.wikipedia.org/wiki/Coefficient_of_determination)) of the prediction.\n", " - for this we compare the observed data to the predicted data" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# predict the training data based on the model\n", "y_pred = lin_svr.predict(X_train) \n", "\n", "# caluclate the model accuracy\n", "acc = lin_svr.score(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "accuracy (R2) 0.9998468951478122\n" ] } ], "source": [ "# print results\n", "print('accuracy (R2)', acc)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now lets plot the predicted values" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 0, 'Predicted Age')" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEHCAYAAAC3Ph1GAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAuIElEQVR4nO3deXxU1d3H8c9vZjLZV0IgLCEoe0BRcUOlGEEJrn3a2qK2bi1ape1D21TUViFqoU1bnrbU1g2XKu4bKCOKIEvFBZQtYSdhDYQl+ySZycx5/riTFCFACEkmk/zerxcvJmfunfldRr4cz5xzrhhjUEopFXpswS5AKaVU82iAK6VUiNIAV0qpEKUBrpRSIUoDXCmlQpQGuFJKhSjHyQ4QkQhgKRAeOP4NY8zDIvIc8C2gLHDobcaY1Sd6reTkZJOenn469SqlVKezatWqg8aYrke3nzTAgVog0xhTKSJhwHIRcQWeyzbGvNHUItLT01m5cmVTD1dKKQWIyI7G2k8a4MZa6VMZ+DEs8EtX/yilVJA1aQxcROwishooBj4yxnweeOoxEVkrIjNFJLy1ilRKKXWsJgW4McZnjBkO9AIuEJGhwP3AIOB8IAm4r7FzRWSiiKwUkZUHDhxomaqVUkqd2iwUY0wpsBgYZ4wpMpZa4FngguOc86QxZoQxZkTXrseMwSullGqmkwa4iHQVkYTA40hgLLBRRFIDbQLcAKxvvTKVUkodrSmzUFKB50XEjhX4rxlj3hORRSLSFRBgNXB365WplFLqaE2ZhbIWOKeR9sxWqUgppRQALpeL3NxcgGGNPa8rMZVSqh1yuVzc+8sp7EgZCVDX2DFNGUJRSinVhup8fn77wkL8Wb8FZ9Rxj9MAV0qpIKsfKikoKKD7WZchI77PoT7WKLV4q497nga4UkoFkcvlYtKkSTjiuhL2rbso6j4MqgDjx77jC+J3LKHwOOdqgCulVBD94U9/QTLGUTtwDNidAMjhQqI2vE/N3k1UO53HPVe/xFRKqSAwxrAwfz+Fg2+mbsh4sDuR2gpi8t4icdWzVO/ZyKxZs0hNTYXjdLa1B66UUm1s+4FKct7L55NNByCmK/h9RO76jMjtS7D5aqlyV5Genk5WVhZZWVmIyLrGXkcDXCml2khlbR1/X7SF2csL8PqsTV0HJ8Kml2dAbQkSFUWV243H4yE7O/ukr6cBrpRSrcwYw7ur9zLdtYH95bUA9EqM5LdXD+GqjG58cJaN3NxcCgsLSU9PJzs7m6ysrJO+rljbfbeNESNGGL2hg1KqM1m/p4ypc/NYuaMEgHCHjXtG9+Oub51BRJi9Sa8hIquMMSOObtceuFJKtYKSKg9/+nATL3+xE3+gn5w1tDsPXj2YXonHX5xzKjTAlVKqBfn8hjlf7OTPCzZRWu0FoH9KDFOvy+CSfskt+l4a4Eop1UK+LDzMw+/mkV9UDkBMuIPJYwfwo4v7EGZv+VnbGuBKKXWa9pXVMN21gXdX721ou3FEL34zbhDJMa13t0kNcKWUaqbaOh+zlxfy90VbcHt8AJzdK55p1w9leO+EVn9/DXCllGqGxRuLmTYvj8JDbgC6RDu5b9wgvnteL2w2aZMaNMCVUuoUFB6s4pH38vl4YzEAdptw68Xp/GJMf+Ijw9q0Fg1wpZRqArenjn8s3sqTS7c3rKIceWYXpl6XwYBusUGpSQNcKaUakZOTw8yZM6moqCBp+BiSx9yFG2tnwB4JEfz26iFkDe2OdV/34NAAV0qpo+Tk5PDII4/gTOlLytX34+yVgRuw4WfSFQP56bfOJNLZtFWUrUmX0iul1FESu/XEMfwGos4eh9is+ds1Wz/H88UrHN65uc3r0aX0Sil1Ej6/4bWVu4j5fi72qHgA6kr2ULnkWTw71+D1eoNc4TdpgCulFLBqRwlT5+axbk8Z9qh4/LVu3F++SfWa+eD34fP5iImJCXaZ36ABrpTq1Iorapjh2shbX+1paOtnP8SyZ38N1WXY7XZ8Ph9+v5/JkycHsdJjnTTARSQCWAqEB45/wxjzsIj0BV4BugCrgB8aYzytWaxSSrUUT52f5z4t4G8fb6Wytg6AoT3jmHZdBuf1SSInvJCZM2dSWVlJTEwMkydP5qGHHgpy1d900i8xxZojE22MqRSRMGA58Avgl8BbxphXRORfwBpjzD9P9Fr6JaZSKphcLhe5ublsd4cTduFNeCOTAEiKdpJ91UBuHNEbexutojwVzf4S01gJXxn4MSzwywCZwE2B9ueBqcAJA1wppYLF5XJx728egnO/A6nD8AIYP6N72vjrj0cTH9W2qyhbQpPGwEXEjjVM0g/4B7ANKDXG1AUO2Q30PM65E4GJAGlpaadbr1JKnbJqj4/75/wH/7gHwW4FtaOkAPvXb7IzRoj/+dVBrrB5mhTgxhgfMFxEEoC3gUFNfQNjzJPAk2ANoTSjRqWUahZjDK71+3j0/XxKe14MgK2mjOgtC3DuzwNjKCwsCXKVzXdKs1CMMaUishi4GEgQEUegF94L2HPis5VSqu1s3l/B1Ll5fLrtkNXgr8Ox9RPid3+G+K353FVuN+np6cEr8jSd9BYRItI10PNGRCKBscAGYDHw3cBhtwLvtlKNSinVZOU1XnLm5ZP112UN4X3FoBSmXRSGWTMXd0UpxhiqqqrweDxkZ2cHueLma0oPPBV4PjAObgNeM8a8JyL5wCsi8ijwNfBMK9aplFKNaphZUlBI8vnXUDvwKioCCyb7Jkfz0LVDuHxgCgApkUJubi6FhYWkp6eTnZ1NVlZWEKs/PboXilIqZLlcLiZNmoQj5Ux8w79DXXwvAJw2mHzlIO64NJ1wR/A3nTpduheKUqrDmT7zH3DBzdT2ubChzb5rFd0PfslPfz8viJW1DQ1wpVTI8fr8vLBiBzuH3gbOSADsFUXEbJyPo3QHu0tCd2bJqdAAV0qFlE+3HuThuXlsKa60wttTRfS2xUTsWYlgQn5myak46SwUpZRqay6Xi8zMTPr27UtmZiYul4vdJW7ueWkVNz39OVuKK7EJXJYKNtcj+Dd/AsbfIWaWnArtgSul2pX6LyadTidJSUns3X+Aex5/D8cyP16/dcyIPolMvS6DoT3jcQ2wdaiZJadCZ6EopdqVzMxMioqKiIqOxtN1EFUDxuGPTAQgJTac+8cP4obhPYN6L8q2prNQlFIhoaCggLjeAygfOB5vl35Wo78ONi1m0Wt/IiZcY6ue/kkopdqNihovcaNuo6zbOWCz5m+HHdyMbfVb9IwL0/A+iv5pKKWCzu83vPX1Hma4NlCWao0USNVBYjYvwLvza7weD9mPzApyle2PBrhSKqjW7i7l4bl5fL2zFIDIMDtje/pZ+/qL7CzY1um+mDwVGuBKqaA4VFlL7oJNvLpyF/VzKa45K5UHxg+mR0Ik3D0+uAWGAA1wpVSbcLlcTJkyhc1bthJ51lXEXXIThFmrKAd2i2XqdRlcfGaXIFcZWjTAlVKtzuVycfvtt1MZlUrSTbmEJfcBwNRW8f2MGH5/+6U47Lqu8FRpgCulWt30v/4LRt5B8oBLADDGT3XeItwrXmbl2UNw/FiHS5pDA1wp1WpqvD6eXradnUNvJcIRDoC3aDOVS2dTV7wdr9dLYWFhcIsMYRrgSqkWZ4zh4w3F5LyXz87DbnCE468qpXz5v/FuXg4Y/H4/YWFhnWbjqdagg05KqWZrbNOp7Qcquf25L/nxCyvZediNwyaM6QWedx7EnbcIn68On89HXV0dcXFxnWbjqdagPXClVLMcs+nUgcPc88SH2Jb68QWmBV7WP5mHrx1Cv5RYXGfarFkomzcDMGTIEGbMmKHzu0+DbmallGqWIzedqu0+jKr+V2LC4wDolRjJb68ewlUZ3TrVplOtRTezUkq1qIKCAuL6DKFs0NXUJVjTAvF5kQ0fsvDtvxMRFvr3omzvNMCVUqespMpDzOgfU5ZyFoj1VZpzfx6y5h16JkRoeLcRDXClVJP5/IY5X+zkzws2UdFtOABSvo/YLR/g3b3euhvOY7rpVFvRAFdKNckXBYd5eG4eG4rKAYiNcDCup4+Vr7zMjoLtuulUEJw0wEWkN/AC0A0wwJPGmL+KyFTgJ8CBwKEPGGPmt1ahSqng2FdWw3TXBt5dvbeh7cYRvfjNuEEkx4TDT3QVZbA0pQdeB/zKGPOViMQCq0Tko8BzM40xf2q98pRSbS0nJ4eZM2dS4a6my8XfI/ai71GHNaZ9du8Ecq7L4OzeCcEtUgFNCHBjTBFQFHhcISIbgJ6tXZhSqu3l5OTwyCOPEHnGCLqPvgNHYg/qgAi85Hz3PL57bi9sNp0W2F6c0jxwEUkHlgJDgV8CtwHlwEqsXnpJI+dMBCYCpKWlnbdjx47TLlop1Tq69BlE2IU/IOKM8wEwfh9Vq+fjWzOXkv17T3K2ai3Hmwfe5AAXkRhgCfCYMeYtEekGHMQaF38ESDXG3HGi19CFPEq1T25PHbMWbeUfCzcijjAAPLvWUbn0WeoO7cLr9eL1eoNcZed1Wgt5RCQMeBN4yRjzFoAxZv8Rzz8FvNdCtSql2ogxhnlri/j9+xvYV16DOMKoKz9A1fIX8Gz7HACfz0dMTEyQK1WNacosFAGeATYYY/5yRHtqYHwc4NvA+tYpUSnVGjYUlTN1bh6fFxwGwOmwMYTdzH/2fxF/HXa7HZ/Ph9/vZ/LkyUGuVjWmKT3wS4AfAutEZHWg7QFggogMxxpCKQTuaoX6lFItrNTtYeZHm/n3ZzvwB0ZQrxzSjd9dM4TeSVHkOIuYOXMmlZWVxMTEMHnyZB566KHgFq0apZtZKdVJ+PyG11buInfBJg5XeQA4s2s0D1+bwagBXYNcnToR3cxKqU5s1Y4Sps7NY92eMgBiwh384or+3DoyHadDbwsQqjTAlerAiitqmOHayFtf7Wlo+865vbhv3EBS4iKCWJlqCRrgSnVAnjo/z31awN8+3kplbR0Aw3rGM/W6DM7rkxjk6lRL0QBXqoNZuvkA0+blse1AFQBJ0U5+feVAvn9+b+y6irJD0QBXKkS5XC5yc3MpKCigb9++3Dbp1yyrTOGjfGuJhk3ghxf14ZdjBxIfFRbkalVr0ABXKgS5XC4mTJhARUUFxhZGae9L2fZpLeKwwvvCvklMvS6DwalxQa5UtSYNcKVC0D333ENZWRlRAy8hMfNOHHEp1hPuEv5+ZybXnJWq96LsBDTAlQpBe6sMKd9/lMj04QCYOi/lX7xJxRdvcu3f3MEtTrUZDXClQkhZtZf/W7iZ7rf9DbFZe3RXb/2cksXP4D2suwV2NhrgSoUAv9/w+qpd/PGDTRyq8iA2O97DeyhZ9BS1BV9Rv6I6Ojo6yJWqtqQBrlQ7cvTMkuzsbLpnXMTUuXms2W2toox22hlsdvDOc7/AX+fBGIOIYLfb+c1vfhPkK1BtSQNcqXbC5XIxadIknE4nSUlJ7D1cwT3PfYrp62845obhPbh//GC6xUWQE7ZfN53q5HQzK6XaiczMTNatW0dJWTlRZ2eRcOlN2MKtIZEhqXHkXJ/BiPSkIFepgkE3s1Kqnfv666+pie9DyvU5OJPTAPBVl+P58g3mLX9DV1GqY2iAK9UO7C5x47z8XuIHXAwE7kW5ZgElS/+Nra5aw1s1SgNcqSCq8fr415Jt/POTbUQGwrtmVx4lHz+Jt3i7NbvEptu9qsZpgCsVBMYYFuTt59H389ldUg2Av+owJYufxb1hScPMEpvNRmxsbJCrVe2V/tOuVCvLyckhMTERh8NBYmIiv5z6R340+wvufnEVu0uqCbMLd3/rTG5N3kHNpmU4HA4iIiJwOByIiN6PUh2X9sCVakU5OTlMmzbN6lE7I5Fzv8ObVQORLQcBGD2wKw9dM4QzusZA1m8JE79ODVRNptMIlWpF0dHRuN3VxAzNJOFbt2GPsW6m4Cvbx7M/u4YrBqfoplPqpHQaoVJBUBfXg+7/czfhPQcB4PfUULbiVSpWvsuYf94Z5OpUqNMAV6oVHKqs5U8fbqL7j/6CiPVVk3vDUkqXPIe3rFh73apFaIAr1YLqfH7+/dkO/vLRZipq6hCx4SkuoOTjJ6ndtb7huPT09OAVqToMDXClWsiKbYeYOjePTfsrAIiLcJDVy8dTT03FU1qCMQabzUZCQgL/+Mc/glyt6ghOGuAi0ht4AegGGOBJY8xfRSQJeBVIBwqBG40xJa1XqlLt097Sah6bv4H31xYBIAI/OD+N7KsGkhTtZHTPf5Obm0thYSHp6elkZ2eTlZUV5KpVR3DSWSgikgqkGmO+EpFYYBVwA3AbcNgYM0NEpgCJxpj7TvRaOgtFdRQul4s//OkvFIafiWSMw9itmwafm5bAtOuGMqxXfJArVB1Js2ehGGOKgKLA4woR2QD0BK4HRgcOex74BDhhgCvVEcyfP59Jv38Cc84tEJ2MAagp59az43n41pHYdN8S1UZOaQxcRNKBc4DPgW6BcAfYhzXE0tg5E4GJAGlpac0uVKn2YPuBSn41dxv+SydaDX4fkTs/w+TNZ9naJGy3jw9ugapTaXKAi0gM8Cbwv8aY8iOnQRljjIg0OhZjjHkSeBKsIZTTK1ep4KisrePvi7Ywe3kB3oQzAAg7tJXoTS4c7oMYp53CwsLgFqk6nSYFuIiEYYX3S8aYtwLN+0Uk1RhTFBgnL26tIpUKFmMM767ey+/nb6C4ohYAR00ptjVvE1tRSH03xu1269RA1eaaMgtFgGeADcaYvxzx1FzgVmBG4Pd3W6VCpYJk/Z4yps7NY+UOa3JVRJiNe0b3o0/1Fn4572vcTidRUVG43W48Hg/Z2dlBrlh1Nk3pgV8C/BBYJyKrA20PYAX3ayJyJ7ADuLFVKlSqjeTk5DBz5kyqvNDl8tsIz7gCAn3s8cO688D4wfRKjAL647SLTg1UQaebWSmFFd6PPPoYMWePI/aSm7BFxACQINU8fudoRvZLDnKFqjPTzayUOoG/vTyflFv+TFhKXwD8tVVUrHiFsi1LGTn9YJCrU6pxGuCqU9tXVsN01wZirv9dQ1t13iKqVszB7y7D6/UGsTqlTkwDXHVKtXU+Zi8v5O+LtuD2+ADwFG2matmz1O3fBoDP5yMmJiaYZSp1QhrgqkNzuVzk5uZSUFBA3759yc7OJqLveeS8l0/BwSoAkmOcDPJs5tU592GzCXa7HZ/Ph9/v19uZqXZNA1x1WC6Xi0mTJuF0OklKSmJPuYefvrwWevgBsNuE20am8/Mr+hMfOZYBjkN6OzMVUnQWiuqwMjMzKSoqIio2AXffUVT3GQk2q89ySb8uTL02g/7d9I7vqv3TWSiq09leUEDM4FGUDLgKf4S1O6CtuhTz1Ru8OP0tvSuOCnka4KpD2lBUTkTWfVTG9bYafF4id/wHk7+AHinJGt6qQ9AAVx1KqdvDzI828+/PduAPhLetaB1xWz+i9vBevB4P2dl/CHKVSrUMDXDVIfj8hle/3EXugo2UuK2522d2jWZ8dzfvf/EhhUW65F11PBrgKuSt2lHC1Ll5rNtTBkBMuINfXNGfW0em43TY+NXNuke36pg0wFXIKq6oYYZrI299taeh7Tvn9uK+cQNJiYsIYmVKtQ0NcBVyPHV+nvu0gL99vJXK2joAhvWMZ+p1GZzXJzHI1SnVdjTAVUhZuvkAU+flsf2AtYoyKdpJ9lUDuXFEb+x6L0rVyWiAq5Cw85CbR9/P58P8/QDYBH50cTqTxwwgPiosyNUpFRwa4Kpdq/b4+OcnW/nX0u146qwl8Bf2TWLa9RkM6h4X5OqUCi4NcNUuGWOYv24fj72fz96yGgBS4yN4YPxgrjkrVRfiKIUGuGqHNu+v4OF381ix/RAATruNn4zqy72X9yPKqf/JKlVP/zaodqOs2sv/LdzMCyt24PNbm6yNGZzCb68eQnpydJCrU6r90QBXQef3G15ftYs/frCJQ1UeAPomR/PQtUO4fGBKkKtTqv3SAFdB9fVOaxXlmt3WKsoop52fZfbnjkvTCXfYg1ydUu2bBrgKigMVtfzhg428sWp3Q9sNw3tw//jBdNNVlEo1iQa4alNen5/nPy3krwu3UBFYRTkkNY5p12dwfnpSkKtTKrScNMBFZDZwDVBsjBkaaJsK/AQ4EDjsAWPM/NYqUnUMy7ccZOq8PLYWVwKQEBXGr68cyIQL0nQVpVLN0JQe+HPALOCFo9pnGmP+1OIVqQ5n12E3j72/gQ/y9gHWKsqbLkzjV2MHkhjtDHJ1SoWukwa4MWapiKS3QS2qg6nx+vjXkm3885Nt1AZWUZ6fnsjU6zLI6BEf5OqUCn2nMwY+SUR+BKwEfmWMKWnsIBGZCEwESEtLO423U6HCGMOCvP08+n4+u0uqAUiJDeeB8YO5fngPXUWpVAtp0l3pAz3w944YA+8GHAQM8AiQaoy542Svo3el7/i2FlcwbV4+y7YcBCDMLtx56RlMyuxHTLh+Z65Uc7ToXemNMfuPeOGngPdOozbVAVTUePnrwi0892khdYFVlKMHduWha4ZwRteYIFenVMfUrAAXkVRjTFHgx28D61uuJBVK/H7DW1/vYYZrIwcrawHo0yWKh64ZwhWDuwW5OqU6tqZMI3wZGA0ki8hu4GFgtIgMxxpCKQTuar0SVXu1bncZD81dz9c7SwGIDLMzKbMfd17al4gwXUWpVGtryiyUCY00P9MKtagQcaiyltwFm3h15S7qv0K55qxUHhg/mB4JkcEtTqlORL9VUk1W5/Pz4mc7+MtHmymvsVZRDuoey9TrMrjojC5Brk6pzkcDXDXJim2HmDo3j037KwCIi3DwqysHcvOFaTjstiBXp1TnpAGuTmhvaTW/n7+B99Za31mLwA/OT+PXVw6gS0x4kKtTqnPTAFeNqvH6eHrZdv6xeBvVXh8A56YlMO26oQzrpasolWoPNMDVNxhj+HhDMTnv5bPzsBuArrHh3J81iBuG98Smm04p1W5ogKsG2w9UMm1ePks2W5tMOmzCHZf25WeZ/YiNCAtydUqpo2mAKypr6/j7oi3MXl6A12fNCxw1wFpF2S9FV1Eq1V5pgHdixhjeWb2H6fM3UlxhraLsnRTJ764ewtgh3XTTKaXaOQ3wTsDlcpGbm0tBQQF9+/YlOzub3meNZOrcPFbusDaRjAizcc/ofkwcdYauolQqRGiAd3Aul4tJkybhdDpJSkpi78Ey7pm9FM70U78P5fhh3Xlg/GB6JUYFtVal1KnRAO/gcnNzcTqdREXHUNNzBLVnZmKcVlAP6BbD1GszGNkvOchVKqWaQwO8A3O5XKxYsQJ76kASLrkNR3I6AOKthvXv8/78ZwjTVZRKhSwN8A7K5XJx768fJH7cL4gYeBkAxvixFX5O+MYF9OgSq+GtVIjTAO+Aaut8PPjiEvxZvyXCYS139+7bQtnipzAHC+nevTvZM3KCXKVS6nRpgHcwizdaqygP97Z63VJbiW3dXA5/8R6emmrsdjuzZs0iKysryJUqpU6XBngH4HK5+P3fnmJX8gXQc5jVaPzYty8nfudybL5aEgf0p6qqitTUVA1vpToIDfAQdOS87tjEZA53Px/H0FvBHvg492/i233qmPvVG1Q7nURFReF2u/F4PGRnZwe3eKVUi9EADzEul4vbb7+diooKHGdehPfiH+GItaYB2qpLid7yAd6ClazZkcqsWbPIzc2lsLCQ9PR0srOztfetVAcipv6eWG1gxIgRZuXKlW32fh3FkT3u4uJi/LHdSRxzF85eGQCYOg9Vq96hT9VmxO/FGENJSQnbt28PcuVKqZYgIquMMSOObtceeDt35ErKhJQelPcfR8zwLMRmLXd3b1lBycdP4ysvJv3cc602t5v09PQgVq2Uagsa4O1Ufa97+fLl+PyGmLOuJP6iHxIbGQuA99AuqpY9R03haupqaxERjDE61q1UJ6IB3o7Uh3Z+fj4lJSX4fD4c3fvTZczdhHfvB4C/1k3pf16mYtU8oiKciAh2u52IiAhKSkp0rFupTkQDvJ3Iyclh+vTp1NXV4fP5sEUlkDD2dmKGZjYcU7nuYyr/8yK1ZQew2Wx4vV7CwsLo2rUrs2fP1tBWqpPRAG8HXC4X06dPx+/344yIxDFkLPEjf4At3Np0qrZoC4cXPoFn70bCw8Ox2WwMHTqUiooK7XEr1YmdNMBFZDZwDVBsjBkaaEsCXgXSgULgRmNMSeuV2TG5XC6mTJnCunXrMMYQkX4OiWPvxpHUEwCfu4zSpS9QtW4hxm/dWNhutzNw4EDWrFkTzNKVUu1AU3rgzwGzgBeOaJsCfGyMmSEiUwI/39fy5XVMLpeLe++9l4KCAgAc8d1IzPwxUQMuBsD4fVR8PZ+yZS/ir61qOC88PJzu3bszY8aMoNStlGpfThrgxpilIpJ+VPP1wOjA4+eBT9AAb5KcnBwee+wxPB4P4ggn7qLvEn/hdxCHE4Canes4/NG/8B7cAYCINNzabODAgcyYMUOHS5RSQPPHwLsZY4oCj/cB3Y53oIhMBCYCpKWlNfPtQt/RwyVRAy8hMfNOHHEpANSVH6Bk8WzcG5cBEBYWRs+ePRERHedWSjXqtL/ENMYYETnuck5jzJPAk2CtxDzd9wtF9cvfDxw4gKNLGkljJhLR52wATJ2X8i/epOyz1zFeaz53fHw8c+bM0cBWSp1QcwN8v4ikGmOKRCQVKG7JojqaKVOmUFJVQ/zlPyb23KuPWEX5OSWLnqKudB9gDZekpKTw7LPPangrpU6quQE+F7gVmBH4/d0Wq6iD8fsNu8J60+32+7BFxQPgPbyHwx8/Sc32Vd84tk+fPjz++OMa3kqpJmnKNMKXsb6wTBaR3cDDWMH9mojcCewAbmzNIkPV1ztLeHhuHnFjfgqA31NN2aevUL7yXfDVNRzncDgYMmSITg1USp2SpsxCmXCcp65o4VpCVk5ODjNnzqSiooLw8HBS0vphzroO+l7UcIw7/xOqPn0JT9mBhvC22Wz06NEDp9OpUwOVUqdMV2KeBpfLxW233UZxceArAJsd+5Cx+C69CVt4tNVWspvx3St5+8sXobocESE8PBxjDImJifTv319nmCilmkUDvJnqZ5bUh3dEn7OtPbqTramSvupy4gqX4tu8hI2p3Zk9e7beXEEp1aI0wE9R/Y6BK1aswOPxYI9LITHzTqIHXgJYqygrV7soXfYS3TIGYKIiKSwsJCsrSwNbKdWiNMCbqH4hTn5+Pk6nE68fYi/+PnEXfhdbWDgANbvyOLzwX3iLCxrO05srKKVaiwZ4E1x++eV88sknDT+H9T2f7pl34kjoDkBdxSFKFj+De8PS/x4TFkZVVZXeXEEp1Wo0wE8gJyeHRx99FK/XC4CjSy+SxtxFZPo5ABifl/Iv3qFsxasYb03DeTabjaSkJFJTU3WsWynVajTAjyMnJ4dHHnmEuro6xBlJwiU3EXvetYjd+iNzb/uS0kVPYas6hPHWYrPZsNvtDB48WDecUkq1CQ3w4/jjH/+Iz+cnemgmid+6HXtMIgDekr2UfPwU1du+RETo27cvs2bN0sBWSrU5DfCA+tklBQUFxMXF4Y1Npdu37ya85yAA/J4ayla8SvmXbzcsxImKitLwVkoFTacP8KNnl3Tv0499aVfQfdzliNgAqMpfQskns/FVHPrGua+//rqGt1IqaDp1gNcvxtm/fz+IjchBmXguu4XIiBgAPMUFHF74BLW71h9z7s0336zhrZQKqk4d4FOmTGH//v2E9x5G0ti7cHZNB8BXXUHpshepXvchGP83zklISGDy5Mk89NBDQahYKaX+q1MH+Na9h0i+7jdEDx4FgDF+KtcsoHTpv/FXlxMVFcWgQYNwu914PB4d71ZKtSudMsBrvD6eWrqd5B/9DalfRbk7n5KFT+DZv63huH79+lFSUqJ7lyil2qVOFeDGGBZuKOaR9/LZediNhIVTV3mY0k+epSpv8TeOtdlsuj+3Uqpd6zQBvv1AJdPm5bNk8wEAHDZhdA/DSw/8nJqK0mOOnzDheNugK6VU+2ALdgEtLScnh8TERBwOB4mJifxu2qNMd23gqv9b2hDel/VP5oP/HcXTk67mrVfn0K1bt4bzbTYbN998My+++GKwLkEppZqkw/TAc3JymD59OjU11p4kNpsNkzaC5w70wb5kOwC9kyL53dVDGDukGyICQFZWFvv27Qta3Uop1VwdIsCP3LcEICzlDJLG3kVErwwAjLeWX40fxsRRZxARZg9mqUop1WJCOsDrl78vWbIEv9+PLSKWhFE/JGb4uP+uoty4jJJFs/n5n4uDXK1SSrWskAzwY+5FKTZizhlPwmU/xB4ZC4DnwA5KFj5Bzc61DcMlSinVkYRcgN9yyy289NJLDT+H98ogacxdOLudAYC/ppLS5XOo+Pp98PuCVaZSSrW6kArwI8PbHtOFxNG3E50xuuH5yrUfUrLkefzusoY2m81GXFxcW5eqlFKt7rQCXEQKgQrAB9QZY0a0RFFHq98xcO3atWB3EDfiBuJHfh+bMxKA2r2bOLzwCTxFm48512azMXny5NYoSymlgqoleuCXG2MOtsDrHMPlcnHvvfdSUGDdJDjijBEkXfETwpJ6AuCrKqFkyfNUrfsYMMecHxERwf33368bTymlOqR2OYRS3+Nev349fr8fR0IqiVf8hKh+FwBg/D4qVs2jdPkcjMd9zPm6Y6BSqjM43QA3wIciYoAnjDFPHn2AiEwEJgKkpaWd9AVdLheTJk2yFtc4wkm46HvEnf9txBEGQHXhakoWPoH30K5Gz582bZoGt1KqUxBjjh16aPLJIj2NMXtEJAX4CPiZMWbp8Y4fMWKEWblyZaPPHdnrNsYQNegyEi6/A0dsMgB1ZcWULHoa9+ZPj1uPLoFXSnVEIrKqse8YT6sHbozZE/i9WETeBi4AjhvgjXG5XNxzzz0UFhYCENY1naQxdxGRNgwAv7eW8i/epPyzNzF1tcecb7fbycjI0DvBK6U6nWYHuIhEAzZjTEXg8ZVAzqm8hsvlYsKECZSVlWGLiCH+0luIPScLsVnL3as2/YeSRc/gKz92FWVYWBiDBw/W4FZKdVqn0wPvBrwdWOXoAOYYYz5oyom33HILr776qrV3idiIOXscCaN+iD0qHgDvoV0cXvgENYWrG84RERwOh4a2UkoFNDvAjTHbgbNP9bwjF+OE9xxE4pi7Ce/eDwB/rZvS/8yhYtW8hlWUCQkJzJkzRwNbKaWOclpfYp7ym1mzVbBHJ5Iw+nZihmY2PFe57mNKljyLv6q0oc3pdPLOO+9oeCulOrVW+RKzOeIu+B/iR/4AW3gUALVFW6xVlHs3frMwh4MHH3xQw1sppY6jTQM8LDmNxMvvAMDnLqN06QtUrv0IjP8bx6Wnp/P4449reCul1Am0aYCLw2mtovzqfcqWv4S/tuqYY3QhjlJKNU2bjoGHJfUyYnfgPbij0ed1IY5SSh3reGPgbXpT47qSPceEt4gQFRXFtGnTNLyVUuoUBG0zq2HDhlnbwyqllGqWNu2Bn3feeRhjMMZoeCul1Glq0wBXSinVcjTAlVIqRGmAK6VUiNIAV0qpEKUBrpRSIUoDXCmlQlRb70Z4AGh8GWbTJAMHW6ic9kavLTTptYWmULu2PsaYrkc3tmmAny4RWdnYctKOQK8tNOm1haaOcm06hKKUUiFKA1wppUJUqAX4k8EuoBXptYUmvbbQ1CGuLaTGwJVSSv1XqPXAlVJKBYRMgItIoYisE5HVIrIy2PWcDhGZLSLFIrL+iLYkEflIRLYEfk8MZo3NdZxrmyoiewKf3WoRGR/MGptDRHqLyGIRyReRPBH5RaA95D+3E1xbR/jcIkTkCxFZE7i2aYH2viLyuYhsFZFXRcQZ7FqbI2SGUESkEBhhjAmluZuNEpFRQCXwgjFmaKDtj8BhY8wMEZkCJBpj7gtmnc1xnGubClQaY/4UzNpOh4ikAqnGmK9EJBZYBdwA3EaIf24nuLYbCf3PTYBoY0yliIQBy4FfAL8E3jLGvCIi/wLWGGP+GcxamyNkeuAdiTFmKXD4qObrgecDj5/H+gsUco5zbSHPGFNkjPkq8LgC2AD0pAN8bie4tpBnLJWBH8MCvwyQCbwRaA/Jzw1CK8AN8KGIrBKRicEuphV0M8YUBR7vA7oFs5hWMElE1gaGWEJumOFIIpIOnAN8Tgf73I66NugAn5uI2EVkNVAMfARsA0qNMXWBQ3YTov9ghVKAX2qMORfIAu4N/K96h2Ssca3QGNtqmn8CZwLDgSLgz0Gt5jSISAzwJvC/xpjyI58L9c+tkWvrEJ+bMcZnjBkO9AIuAAYFt6KWEzIBbozZE/i9GHgb64PoSPYHxiLrxySLg1xPizHG7A/8JfIDTxGin11gDPVN4CVjzFuB5g7xuTV2bR3lc6tnjCkFFgMXAwkiUn9P4F7AnmDVdTpCIsBFJDrw5QoiEg1cCaw/8VkhZy5wa+DxrcC7QaylRdUHXMC3CcHPLvBl2DPABmPMX454KuQ/t+NdWwf53LqKSELgcSQwFmuMfzHw3cBhIfm5QYjMQhGRM7B63QAOYI4x5rEglnRaRORlYDTWjmj7gYeBd4DXgDSsHRtvNMaE3JeBx7m20Vj/G26AQuCuI8aNQ4KIXAosA9YB/kDzA1hjxSH9uZ3g2iYQ+p/bWVhfUtqxOqyvGWNyApnyCpAEfA3cYoypDV6lzRMSAa6UUupYITGEopRS6lga4EopFaI0wJVSKkRpgCulVIjSAFdKqRClAa6CTkR8gd3u1ovI6yISdRqv9ZyIfDfw+GkRGXKCY0eLyMhmvEehiCQf57nhImJEZNypvq5Sp0oDXLUH1caY4YHdCz3A3Uc+ecSKuVNijPmxMSb/BIeMBk45wE9iAtaOdxNa+HWVOoYGuGpvlgH9Ar3jZSIyF8gPbEiUKyJfBjZXugusVYQiMktENonIQiCl/oVE5BMRGRF4PE5EvgrsC/1xYNOmu4HJgd7/ZYFVe28G3uNLEbkkcG4XEfkwsJ/004A0VnhgReP3sLaYHSsiEUc897tAjctF5GUR+XWg/UwR+SCwSdsyEekw+3So1tesno1SrSHQ084CPgg0nQsMNcYUBHagLDPGnC8i4cB/RORDrJ3zBgJDsHYCzAdmH/W6XbH28hgVeK0kY8zhwD7QDftdi8gcYKYxZrmIpAELgMFYq0mXB1bwXQ3ceZxLGAkUGGO2icgnwNXAmyJyPvAd4Gys7Uy/wtpzG6x7M95tjNkiIhcCj2NtdarUSWmAq/YgMrDdJ1g98GewwvALY0xBoP1K4Kz68W0gHugPjAJeNsb4gL0isqiR178IWFr/WidY6j4GGGJ1pAGIC+zQNwr4n8C574tIyXHOn4C1PJvA7z/C2iDqEuBdY0wNUCMi86Bh97+RwOtHvGf4cV5bqWNogKv2oDqw3WeDQKBVHdkE/MwYs+Co41ryNl824KJA0B5dywmJiB2rl329iDyIVW+X+k3YTvB+pUdfu1JNpWPgKlQsAH4a2PYUERkQ2JlyKfD9wBh5KnB5I+d+BowSkb6Bc5MC7RXAkQH7IfCz+h9EZHjg4VLgpkBbFtDYjQ2uANYaY3obY9KNMX2wet/fBv4DXCvW/RljgGsAAntuF4jI9wKvLSJy9qn8oajOTQNchYqnsca3vxLrhslPYP0f5NvAlsBzLwArjj7RGHMAmAi8JSJrgFcDT80Dvl3/JSbwc2BE4EvSfP47G2Ya1j8AeVhDKTsbqW8C/90xs96bwARjzJdY286uBVxYu/6VBY65GbgzUFce1i3alGoS3Y1QqTYgIjGBG+tGYfXoJ9bfh1Kp5tIxcKXaxpOBRUURwPMa3qolaA9cKaVClI6BK6VUiNIAV0qpEKUBrpRSIUoDXCmlQpQGuFJKhSgNcKWUClH/D5KvRZcRoN7ZAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.regplot(y=y_pred, x=y_train, scatter_kws=dict(color='k'))\n", "plt.xlabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now thats really cool, eh? **Almost a perfect fit**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "... which means something is wrong\n", "- what are we missing here?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "- **recall**: We are still using the test data sets." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\"logo\"
\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Train/test stratification\n", "\n", "Now lets do this again but we'll add some constraints to the predriction\n", "- Well keey the 75/25 ratio between test and train data sets\n", "- But now we will try to keep the characteristics of the data set consistent accross training and test datasets\n", "- For this we will use something called [stratification](https://en.wikipedia.org/wiki/Stratified_sampling)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# use `AgeGroup` for stratification\n", "age_class2 = info.loc[y_train.index,'AgeGroup']\n", "\n", "# split the data\n", "X_train2, X_test, y_train2, y_test = train_test_split(\n", " X_train, # x\n", " y_train, # y\n", " test_size = 0.25, # 75%/25% split \n", " shuffle = True, # shuffle dataset before splitting\n", " stratify = age_class2, # keep distribution of age class consistent\n", " # betw. train & test sets.\n", " random_state = 0 # same shuffle each time\n", ")" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Let's re-fit the model on the newly computed (and stratified) train data and evaluate it' performace on an (also stratified) test data\n", "- We'll compute again the model accuracy (R-squared) to evalueate the models performance,\n", "- but we'll also have a look at the [mean-absolute-error](https://en.wikipedia.org/wiki/Mean_absolute_error) (MAE), it is measured as the average sum of the absolute diffrences between predictions and actual observations. Unlike other measures, MAE is more robust to outliers, since it doesn't square the deviations (cf. [mean-squared-error](https://en.wikipedia.org/wiki/Mean_squared_error))\n", " - it provides a way to asses \"how far off\" are our predictions from our actual data, while staying on it's referential space" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "from sklearn.metrics import mean_absolute_error\n", "\n", "# fit model just to training data\n", "lin_svr.fit(X_train2, y_train2)\n", "\n", "# predict the *test* data based on the model trained on X_train2\n", "y_pred = lin_svr.predict(X_test) \n", "\n", "# calculate the model accuracy\n", "acc = lin_svr.score(X_test, y_test) \n", "mae = mean_absolute_error(y_true=y_test,y_pred=y_pred)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Lets check the results" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "accuracy (R2) = 0.6593855081680796\n", "MAE = 3.2059201603105882\n" ] } ], "source": [ "# print results\n", "print('accuracy (R2) = ', acc)\n", "print('MAE = ', mae)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 0, 'Predicted Age')" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAwRklEQVR4nO3deXjc1X3v8fd3NkkzXiTZsq0FY+FCsMMiE5OQQIQxYREk2EpSCu1tSDF10hvnJm3qNsu9JHVv++TGSdPe6zYNsUmgT9ZeYrMkAgIJEeQmEIONMTgBjDcttmRrtWafOfeP+Y0sy5rRjDT7fF/P48fSb7YzDD7f35xzfp8jxhiUUkqVH1u+G6CUUio/tAAopVSZ0gKglFJlSguAUkqVKS0ASilVphz5bkA6Fi5caJYtW5bvZiilVFF58cUXTxpj6iYfL6oCsGzZMnbv3p3vZiilVFERkSNTHdchIKWUKlNaAJRSqkxpAVBKqTKlBUAppcqUFgCllCpTWgCUUlnT0dHB2rVraW5uZu3atXR0dOS7SWoCLQBKqazo6Ohg06ZN9Pb2UltbS29vL5s2bdIiUEC0ACilsmLr1q24XC48Hg8igsfjweVysXXr1nw3TVm0ACilsuLQoUO43e6zjrndbg4fPpyfBqlzaAFQSmVFc3MzXq/3rGNerxeNcykcWgCUUlmxefNmgsEgY2NjGGMYGxsjGAyyefPmfDdNWbJeAESkUkReEJGXReRVEfk763iziDwvIm+KyA9FxJXttiilcqetrY1t27ZRX1/P4OAg9fX1bNu2jba2tnw3TVkk23sCi4gAHmPMaRFxAs8BnwL+CvixMeYHIvLvwMvGmG8ke67Vq1cbDYNTSqn0iMiLxpjVk49n/RuAiTlt/eq0/hhgLfB/reMPAOuz3RallFJn5GQOQETsIrIX6AN+BhwEhowxYesuXUBjgsduFJHdIrK7v78/F81VSqmykJMCYIyJGGNagCbgncDFaTz2PmPMamPM6rq6c/YzUEopNUM5XQVkjBkCfgG8G6gWkfiGNE1Ady7bopRS5S4Xq4DqRKTa+rkKuAE4QKwQfNi6213Aw9lui1JKqTNysSVkPfCAiNiJFZwfGWMeE5HXgB+IyP8E9gA7ctAWpZRSlqwXAGPMPmDVFMffIjYfoJRSKsOiUcOIP0QgHE14n6LaFF4ppVRyxhiGfSGGfSEiUUOl057wvloAlFKqBBhjGPGHGfaGCEcTn/VPpAVAKaWK3OlAmMGxIKFIah1/nBYApZQqUv5QhFNjQQKhyIwerwVAKaWKTCgSZXAsyOlAePo7J6EFQCmlikQkahjyBhnxh8lEkKcWAKWUKnDGGEZ8YYZ8QSLRzCU464YwSqmS0NHRwdq1a2lubmbt2rUls/n8qD/EsQEfp8YCGe38QQuAUqoEdHR0sGnTJnp7e6mtraW3t5dNmzYVdRHwBsN0DXrpHw2kvKwzXVoAlFJFb+vWrbhcLjweDyKCx+PB5XKxdevWfDctbYFwhN5hH8eH/QSTXMWbCToHoJQqeocOHaK2tvasY263m8OHD+enQTMQjkQZ8AY57Z/dyp506DcApVTRa25uxuv1nnXM6/WybNmy/DQoDZGo4dTpAMcGfTnt/EELgFKqBGzevJlgMMjY2BjGGMbGxggGg2zevDnfTUsoai3pPDbgZdgXysiyznRpAVBKFb22tja2bdtGfX09g4OD1NfXs23bNtra2vLdtHPEw9qODXoZGAsSzWLH39nZyR133AFw6VS36xyAUqoktLW1FWSHHzeTsLbZ6OzsZMuWLThjp/lTji1pAVBKqSzKdccft33HDpxOJ+4KZ8L7aAFQSqksyFfHH9d1rIvq6vkQTTyxrAVAKaUyKN8df1zTeU309/Ul/QaQi03hzxORX4jIayLyqoh8yjr+JRHpFpG91p9bst0WpZTKlvHJ3QEfp05n7+rdVN2zYQOhUOic5bET5WIVUBj4jDFmJXAV8AkRWWnd9nVjTIv156c5aItSSmVU7Iw/RNdgYXT8ca2trdx7770sXrwYEoz25GJT+F6g1/p5VEQOAI3Zfl2llMq2sUCYgRnsxJUrra2t3Hj9dTTWuF+Z6vacXgcgIsuAVcDz1qFNIrJPRO4XkZoEj9koIrtFZHd/f3+umqqUUgkFwhF6hnycGPEXbOcfNzAWTHhbzgqAiMwBHgI+bYwZAb4BLAdaiH1D+NpUjzPG3GeMWW2MWV1XV5er5iql1DlCkSh9o366B334Z7gNY64c6B3hH396gPZ/+1XC++RkFZCIOIl1/t81xvwYwBhzYsLt3wIey0VblFIqXeFIlEFviNOBzOzElS3BcJRnXu9n555ufn98dNr7Z70AiIgAO4ADxph/mnC83pofAGgH9me7LUoplY5wJMqQL8RohrZgzJb+0QCPvNzDT/b1MuQLjR9fXufh9tXn8fH/NfXjcvEN4GrgT4FXRGSvdezzwJ0i0gIY4DDwsRy0RSmlplUMZ/zGGPZ1D7NrTw/PvtFPfLMwu0245g8W0r6qgUsb51PlcvDxBM+Ri1VAzwEyxU267FMpVVCiUcOQL5S3dM5U+EMRnj7Qx6693RzsHxs/XuN28v7L6nn/ZQ3Uza1I6bn0SmClVNmLX7075M3spuuZ1DPk4+G9PTz+6nFGJ+wbsKJ+Lu2rGmm9sA6XI711PVoAlFJlbdQfYsgbKsjlnMYYdh8ZZNeeHn7z1inipclpF6572yLWr2rg4iXzZvz8WgCUUmVpLBBm0Buc0b67nZ2dbN+xg65jXTSd18Q9GzbQ2tqasbZ5g2GeePUED+/t4ejAmSiHujkV3NZSz62X1lPtds36dbQAKKXKymw6fpiQs+90Ul09n/6+PrZs2cK999476yJwdMDLrj3dPPnaCbzBM9cZXN40n/ZVjVz9Bwux26aaUp0ZLQBKqbLgC0YY8AYJzPICrvGcfbcbiG0+7/V62b5jx4wKQCRqeP7QKXbu6eHFI4PjxyscNm5YuZj1LQ1cUDdnVm1ORAuAUqqkBcIRBsdCeIOZ2XB9PGd/gqqqKrq7utJ6nhFfiI79x3nk5R56h/3jx+vnV7K+pYGbL1nC3MrEUc6ZoAVAKVWSQpEog94gp/2Z6fjjxnP2rW8AAD6fj8amppQef7DvNDv3dvP0gT4CE4ahrlxWQ/uqRq5cVpvRYZ5ktAAopUpKJGoY8gYZydLVu/ds2MCWLVvwer1UVVXh8/kIhULcs2FDwseEI1Gee/MUO/d080r38Phxt8vOzW9fwm0tDSytdSd8fLZoAVBKlYT4hixD3hDRLF7EFc/Z375jB91dXTQ2JV4FNOgN8ti+Xh59uYeTp8+kci6tdbO+pYEb374Ytyt/3bAWAKVU0Rvxhxgay90WjK2trUknfA/0jrBzTze/fL2fUCRWjAR4z/IFrF/VyBVLq4nFpOWXFgClVNEqpA1ZEiVxzq10cMslsWGe+vlVeWzhubQAKKWKjj8UYWAsWBCZ/MmSONtXNbL24kVUOu15bGFiWgCUUkUjGI4yMBbM2JLOmYonce7c081zb5w8K4mz9cKFrG9p5JLGeQUxzJOMFgClVMELhqMM+TK/pDMu1WgHfyjCU1YS51uzTOIsBFoAlFIFKxiOMuQNcjqQvTP+VKIdeodjSZwd+zOXxFkItAAopQpOLjr+uETRDt/asQP30kvYmaUkzkKgBUApVTBy2fHHTY52MHYXtuZL6V58KX/z0CvjxxfOcbGupYFbLq2nJgNJnIVAC4BSKu/y0fHHxaMdKmobCNRfTmDRCrCf6eCzlcRZCHKxKfx5wIPAYmL7/95njPkXEakFfggsI7Yn8O3GmMFEz6OUKj2BcIRha+/dfIhEDe/90N18/9cHCSxcPuGGEO9Y4uTjbe9geZaSOAtBLr4BhIHPGGNeEpG5wIsi8jPgo8DTxpgvi8hngc8Cf5uD9iil8swfijDsCzGWp47/7CROIN75jw0wf/gNPn7ru7hp7bV5aVsu5WJT+F6g1/p5VEQOAI3AOmCNdbcHgGfQAqBUSfOHIgx5MxfNnK5kSZzrWxp5Z3PukjgLQU7nAERkGbAKeB5YbBUHgOPEhoiUUiXIH4ow6A3iC+b+yt1kSZw3vX0J6/KUxFkIclYARGQO8BDwaWPMyMQr5IwxRkSmjO8TkY3ARoClS5fmoqlKqQzJZ8df6EmchSAn715EnMQ6/+8aY35sHT4hIvXGmF4RqQf6pnqsMeY+4D6A1atXZy/jVSmVMb5ghCFffjr+REmc716+gPUtDbzj/JqCj2jIlVysAhJgB3DAGPNPE256BLgL+LL198PZbotSKrt8wdgZf7ohbalGMSQST+Lctaeb302RxLmupZEl8yvTalM5kGzsmHPWC4hcAzwLvALEZ10+T2we4EfAUuAIsWWgA8mea/Xq1Wb37t1ZbK1SaiZm2vHD2VEME3fYmhjFkEgxJ3HmSqXTTmON+0VjzOrJt+ViFdBzxL6BTeX6bL++Uip7fMEIA94ggVnEMieKYti+Y8eUBSCexLlrTw/PvtE/nsRpE2i9sI72VcWRxFkIynsGRCk1I5mc3J0cxQBQVVVFd1fXOa/5tJXEebAEkjhzwWGzUZXkG5AWAKVUykKRKINjmY1siEcxxL8BAPh8PhqbmgDoGfLxyMvnJnFevCSWxHntRcWZxJkNIkKlM9bpV7nsVDiSD39pAVBKTSsaNQz5Qgz7QmR63vCeDRvYsmULXq93fA4gGApx3Yfv5gs795dsEmem2G2C2+XA7bJT5bRjS+NCtqxPAmeSTgIrlVvGGEZ8YYZ8QSLR7PUV46uAevuovuRa7MvfzSn/mdvr5lRwW0t9SSVxzobTbsNTEev0U5nkFpH8TAIrpYpPrjr+uGWXrGb1R+oZeO0EQ8EIWJ3/ZVYS5zUlmMSZrkqnHbfLjtvlyNiQlxYApdS4aNQw4o8N9WS7449EDc8fOsXOPT28eORMEHCFw8b7Vixm/aqGkk7inI7dJlRZHX6V056VAqgFQCmV047/7CTOM+M89fMrWdfSQNslS5hb6cxqGwpVhdOO25rAzcX1C1oAlCpjuRzqSZTE+Y7za2hf1cC7mheU3TCP3SbjK3bcLkfO378WAKXKkDGGEX+YYW+IcDQ6/QNmSJM4z5Xrs/xktAAoVUaiUcOoP/tn/ANjQX7ySuIkzhtWLsZTUR7dj8Nmo8oV6/CzNZY/U+XxCSilGPGHGBzLbsevSZxgE6HSeabDL+SL1LQAKFXi/KEIJ08HCIazM9QTT+Lcuaeb30+RxHlbSwP186uy8tqFwmm3jS/RrHTaiqbIaQFQqkSdDoQZ8YVmlNCZinJP4qxw2vG4UotcKFRaAJQqIfEx/hF/iFAk82f80yVxrl/VwKWN84vmDDgdNpHxsXy3047DXrhDO6nSAqBUCQhHooz4Y2f80SzEu/hDEZ6ykjjfmpTEeetl9XygRJM4i3VoJ1VaAJQqYuFIlCFfiFF/OOMhbQC9wz4e3ntuEueK+rmsbym9JE6Rievy7ThL4Cw/GS0AShWhcCTKoDfE6UDmO35jDC8eGWTnnp4pkzjXtTSwor50kjjjZ/nxVTuldpafjBYApYpIJGoY9AazcsbvDYZ54tUTPLy3h6MD3vHjC+e4uO3yBm69rDSSOMvtLD8ZLQBKFYFs5vEfHfCya083T752Au+EHb4ub5rP+hJJ4nTabeMdfrmd5SeT9QIgIvcD7wf6jDGXWMe+BPw50G/d7fPGmJ9muy1KFZtsZfWUehJnfGcst9NBlauwL8bKp1x8A/gOsA14cNLxrxtjvpqD11eq6BhjGA3EsnoyuZyzlJM445ELM9kZq1xlvQAYYzpFZFm2X0epUpCtkLZESZyrz6+hfVUj72yuLbphHhGhwnFmArdYL8bKp3zOAWwSkY8Au4HPGGMGp7qTiGwENgIsXbo0h81TKnfi6/hH/ZnL4y/FJE6HzUalyxbbA1fP8mctJ3sCW98AHpswB7AYOAkY4O+BemPM3dM9j+4JrEpNIBxh2FrOmSmD3iCP7UucxHnj2xfjdhXP+o/4Voh6lj9zBbUnsDHmRPxnEfkW8Fg+2qFUvgTCEYa8IcYy2PEf6B1h194envl9X1EnceZiK0QVk5cCICL1xphe69d2YH8+2qFUrvlDsY7fG8xMxx8MR/mllcT5uyJO4iykTVLKSS6WgX4fWAMsFJEu4IvAGhFpITYEdBj4WLbboVQ+Zbrj7x8N8Oi+WBLnoLf4kjjzvRWiipm2AFjj9f8INBhj2kRkJfBuY8yOVF7AGHPnFIdTeqxSxc4fijDoDeILzj6SOVkS53svrKO9wJM4nXYbngoHbj3LLxipfAP4DvBt4AvW768DP0Q7caUSymTH7w9FeNpK4jxYZEmc8U5/ToVDL8YqQKkUgIXGmB+JyOcAjDFhEcnODhNKFTlfMMKQLzMdf6IkzouXzKV9VeEmcdptMt7p65l+YUulAIyJyAJi4/WIyFXAcPKHKFVevMEwQ97Z776VLIlzzdsWsb5AkzhtIrgr7MypcGjWThFJpQD8FfAIsFxEfgXUAR/OaquUKhJjgTBDvhCBWXb88STOXXu6OTboGz9eyEmc8VTNOZUOPC7t9IvRtAXAGPOSiFwLvI3YsuLfG2NC0zxMqZJljOF0IHbGP9ucnqMDXh7e28MTrx4/K4nzsqb5tBdoEmfleKevq3eKXSqrgD446dBFIjIMvGKM6ctOs5QqPPH9dod9s8vpiSdx7trTw+6pkjhbGli+qLCSOF0OG3MrnHgqSmMvXBWTyhDQBuDdwC+s39cALwLNIrLFGPMfWWqbUgUhU/vtjvpjSZwP7z03ifO2y2NJnPOqZpbE2dnZyfYdO+g61kXTeU3cs2EDra2tM24r6AqecpBKAXAAK+LxDdZ1AQ8C7wI6AS0AqiQFw1GGfbPfdvFg/2l27enhqQMnspLE2dnZyZYtW3A6nVRXz6e/r48tW7Zw7733pl0EdAVPeUmlAJw3MbsH6LOODYiIzgWokpOJq3bjSZy79nazryu7SZzbd+zA6XTidseez+124/V62b5jR0oFIJ6jP6citnmKKh+pFIBnROQx4D+t3z9kHfMAQ9lqmFK5NhaIje/PZilnPpI4u451UV09/6xjVVVVdHd1TXn/+G5ZVc7YFbl6pl++Uvk/8RPAB4FrrN93A4uNMWPAddlqmFK5ElvREyQYnnpiN5Xx9d8dH2HnnnOTOK+6YAHtq7KbxNl0XhP9fX3j3wAAfD4fjU1NsXaI4HLEOvxYp2/TJZsKSG0ZqBGRt4CrgD8EDgEPZbthSmVTqks5k42vX/WeaxImcbZdEhvmyUUS5z0bNrBlyxa8Xi9VVVX4fD5CAT+f/NgGFs+r1O0RVUIJC4CIXATcaf05SSz/R4wxetavilYkahjxhRhJceetqcbXxyJ2vvbYHhyvOs5K4rygzkN7SyPXr8htEmdrayv33nsvO751H12HD7KscQl/89efp62tLWdtUMUp2TeA3wHPAu83xrwJICJ/mZNWKZVhgXCEYV+IsUAkrRU98fF1A4TnNhCov5zQguUExQbeUN6TOOOrdu647SY++qFbcvraqvglKwAfBO4AfiEijwM/IDasqVTR8AZjE7szDWdrWHo+ffY6zPlXEvHUjR+3hXz8yXvfxvvzkMQZ3zFLc3fUbCUsAMaYXcAua7XPOuDTwCIR+Qaw0xjzZE5aqFSa4uP7w74QT/38mRldINU77OORvT0MtNxFeELtkOEeOPwCn797PWuvbs7iuzib027D7bLj0fX5KoPS2hReRGqITQT/kTHm+qy1KgHdFF4lY4xhxB9m2BuLapg4gTs+ORoKJbxAKp7EuWtvD78+eCaJ0y5QMfAm3teeoWmuLSNX2aYivhm626VX4qrZSbQpfFoFIN+0AKipJMro+chdd52zPNLr9VK3aBEPPvDAmWN5TuI8s8z0KEsblvDpTX/BulvbNGhNZUyiApCLPYHvB94P9BljLrGO1RJbVbSM2J7AtxtjBhM9h1JTCYQjjPrDnPaHp8zome4CqWMDXnZNkcR5edN81ucoifO5ZzvZ8j8+j8tmqK1w0H/0TTZ/+pNUObbpKh6VdVn/BiAircBp4MEJBeArwIAx5ssi8lmgxhjzt9M9l34DUJD6xO5U3wDGvD7mLr+C5jW3T53EuaqB5XXZTeJ02Gx4KmLj+bfcdAO9vb14PJ4zbRwbo76+np///OdZbYcqH3n7BmCM6RSRZZMOryOWKgrwAPAMMG0BUOXLGMNoIDa+n2oG/8QLpCrmVDNWexGRphZOuWs5ZXX+9fMrWdcSS+Lc88Kv+bu//oeMJmrGJdoQ/dChQ9TW1p51X7fbzeHDhzPyukolk5M5AKsAPDbhG8CQMaba+lmAwfjvUzx2I7ARYOnSpe84cuRI1turCkcoEmXEF2I0wTDPdH70+C/5j87fM1Z9AdjPRC1PTuJMd8I4lXiIiWf6iVburF27Vr8BqKzL6yRwsgJg/T5ojKmZ7nl0CKh8+EPxC7fST+RMlsR548rFrF/VeE4SZ6oTxkDSYnHdmmtxuxzMrUxtuWZHRwebNm3C5XKNp3gGg0G2bdM5AJU5eRsCSuCEiNQbY3pFpJ5YxLRS+IIRhnzBGV24NZskznQSNc+Jh6iqwhcO8J1v/DN3fbAtrQuz2tra2LZtG1u3buXw4cMsW7aMzZs3a+evciJfBeAR4C7gy9bfD+epHaoATLxwK1EiZzKJkjjfvXwB7asauWJp9bSd8nSJmhPFisU8JBxAIkGIBPHYDUffenNGV+W2tbVph6/yIhfLQL9PbMJ3oYh0AV8k1vH/SEQ2AEeA27PdDlV4IlHDaX+YEX/6m6sHw1F++Xo/u/Z2c6D37CTOWy5Zwm1pJnFOmagZCnHPhg3j9xERqpx2li2u5nj3MeZ4Jq4u8rJs2bK03oNS+aYXgqmcm81Wi/2jAR7d18NP9vWmncQ53cRt/Pburi4am2K3X3vttVQ57bgr7HhcDuw20XF7VXT0SmCVdzOd2DXG8Er3MDv39PDsG/3EU5xtAq0X1rE+hSTOdFb5xLdI9FTYE4atdXR06Li9KhpaAFTeeIOxjVfS3WrRH4rw9IE+du3t5mD/2PjxGreTWy+r5wNpJHFOt8pHw9ZUKSu0VUCqxM3kwq24eBLnT/cfZ9R/5tvCxUvm0r6qkWsvqks7HG3KVT6VFXQfeoOG6irt9FVZ0gKgMirdHbfiEiVxEo1Az34W+Y5wx5XraF25eEbtGl/lU1WFRIJIOIB3dIjm+np+8dSTbN26lUOHDtHc3KzDOapsaMZsEevo6GDt2rU0Nzezdu1aOjo68taWYDhK/2iAowNeBr3BlDt/bzDMzj3d/Nl3dvM3D73C/7M6/7lOkNd/gevZf6X6yDMMH3mNLVu20NnZmXbb7Dbhkx/bQGjkJL6Tx5DAKN7RIYLBIGvWrGHTpk309vZSW1tLb28vmzZtytp/y0L6zJTSOYAiVSgrUWY6sXt0wMvDUyRxXtY0n/Utjez4x82c7DuR0pW5ibhdDuZVOcYv/ppq4nbr1q05i2IolM9MlR+dBC4x+c6QGbMu3EpnYjcSNbxwaICde7qnTeJcu/Z6qqvPXtljjGF4eJinn3464Ws4bDbmVMaiGJz26b/gNjc3U1tbe87rDA4O8tZbb6X83lKR789MlS+dBC4x+UiRnOnE7qg/RMf+4zy8t4feYf/48YlJnHMrnWc9Jp0rcwE8FbFOP909cpubm8/plL1ZuqhLkz9VodECUKRy2XHNdGL3YP9pdu3p4akDJwhMiHhYfX4N61c18K7mBQk3XEnlyly7TZhb6WRepQNHCmf7U9m8eTObNm0COGtYZvPmzTN6vmRy+ZkplQqdBC5SmzdvJhgMMjY2hjGGsbGxjHdcvmCEvhF/WhO74UgsouHTP9zLnz/4Ij95pZdAOIrbZad9VSPf+bMr+cqHL+M9y5PvttXa2sq9995L3aJFDA8PU7do0fhFWy6HjYVzK1ha66bW45px5w9nwtjq6+sZHBykvr4+a2PyufjMlEqHzgEUsWxcjRqORDkdCDPqD6c1zDPoDfKTfb08MoMkzlTNqXAwr8pZ1Gv29QpilQ86CayS8gZjnb43GEkrnydZEuf6lgbecX7NjBIy40SEORUOqt3OlCZ1lVLn0klgdY5QJDq+qXo4mvrZfrIkzrZLlrAuzSTOqYgIcysdVFc5ZzXEo5RKTAtAmYlGDaeDsU4/3Wye2SRxpip+xl/j1o5fqWzTAlAmfMEIo4EQY4H0hngykcSZCpfDxtwKJ54Ku3b8lvh8gUZUqGzRAlAgsvGPPRwf4gmkN6ELmU3iTMZT4WB+kU/sZsPEq4YnRlToVcMqk3QSuABkMiLAGIM3GLEmdNPfUD0bSZxTcbsc1HicVDi045+KXjWsMinRJLB+1y4AW7duxeVy4fF4EBE8Hg8ul4utW7em/ByhSJSBsSBHB7ycGPGn1fkbY9h9eID/vms//2X7C/xwdxej/jBOu3DDysX86x+v4t/+5ApuWLl4Vp2/iDCn0sH+Fzq5qfVdVM+dQ1VVFZdffnlOQtGKKYjt0KFDZ10FDXrVsMo8/QZQAGaaRxONGsaCsSEeXzC9CV2ILf184tUTPLy3h6MD3vHjC+e4uHR+kN89+T16D7855faJ6bCJML/KybwqJ08+8Th33303J0+exG63W+8jSm1tLd/+9rezNrxRbEFs+g1AZVJBfgMQkcMi8oqI7BWR0uvZU9Tc3IzX6z3rWLKIAF8wQt+onyMDXvpHA2l3/kcHvPzvp9/g9m/+hv/z8zfHO//LmubzxQ+s5C/eFuSX2/+egd6jVFfPp7+vb8oo5s7OTj5y112sXXs9H7nrrnNut9uEWo+LpbVuajwu7DZh69atjIyM4HA4sNvt2O12bDYbo6OjaX3jSVcmvmXlkl41rHIhr98AROQwsNoYczKV+5fqN4BUzk6D4dgVuumu2Y9LJ4lzuu0TIfkeu9etuTZ2xl/pxDYp7qG5uZnu7m6czjPhb8YYIpEIjY2NGU/gnPi6uUr9zBS9alhlil4IVsDieTST/7HfdNPNjPhDM1qzHzeTJM4pt0+sqqK7q2v89+07duB0OseLRKxwjfHtb27jI+1t53T8cc3NzfT19RGNRrHZYl9Ao9EoDocjq6FoxRjE1tbWph2+yqp8FwADPCkiBvimMea+yXcQkY3ARoClS5fO6EWKYT31xH/s8TX7Rwa8aa3ZnyhREuc7zq+hfZokzlSimM8qEsYgYT8e4+fYmz0JO3+IDW1MNQdQXV2d1eGNXKZ+KlUs8r0K6BpjzBVAG/AJETlnltEYc58xZrUxZnVdXV3aLxAfXsnVln8zFQhHOHU6wNFTXnqHfZz2h9Pu/KdN4vzolWxNIYnzng0bCIVCeL2xAuT1es+JYm46rwmfz4eE/dj8Q9hCXnzesWnPqNva2rj//vtZuXIlIoKIsGLFiqxOAMdfN1epn0oVi4JZBSQiXwJOG2O+mug+FRUV5uqrr07rDH62qymy+e1hpsmbkw16gzy2r5dHJyVxnldTxfpVjdy4cjGeivS+7HV2drJ9xw66u7pobDp7FZCI8MKvfsnnPvMpKpyOolhVo1Q5K7g0UBHxADZjzKj188+ALcaYxxM9xuPxmKVLl6bV0cxm8i8bSwdnu3RzokRJnFddsID2VbNP4pzKnEoHte5YBr9OUiZWDMOOqnwUYgG4ANhp/eoAvmeM+Ydkj/F4PGbFihVpncHP5htAptZiR6IGbzDMWCCCL5ReFs9kwXCU+x59lsde6SPoWTR+PJNJnFOpdNqp9bg0siEFxXbNgSp9BVcAZiJeANJZvjebf4wz/fZgjMEfiuINhvGFIgTDMx/eiesfDfDYvh5+vPsoYxMu8pXRPjj8PJ+76wO877prZ/06kzntNmo9rrSHkMqZXsSlCk1JLQNNZ/leoiWWqQ4fpbp0MH6W7w1G8AUjRDNQWONJnLv29PDsmyfPbMkYjeIcOEjF8ZdxjPTg83p58Dv9GS0ATruNGo+LOdrxp003f1fFouj+dc/kisiZrqeebulgOBJlLBBhLDjzdfpTSZbEOfjy08wbeh176MyVw5PX6M+Gw2aj2uNkboUj4/MH5aIYrzlQ5Snfy0DTEg6Hc7p8L9HSwWuvv4ETI36ODfo4NRbIWOd/fNjPN395kD+67zd87Wevj3f+Fy+Zy+faLub7f34VTb43CQyffeH05DX6M2G3CQvmVHBebRXzKp3a+c+CxjioYlFUcwD5ioIIhCP4g1H84Qj+UOTMUEwGGGN48cggu/b28OuDp4g/s9MurHnbIta3NLCift74/ZNFMMwkrM1uk4SxDWrmdIWUKiQlMQmcqwIQn8SNL9WcSfbOdLzBME++eoJdUyRx3nZ5A7deVk+N2zXlY5Ot0U9VPKFzfpV2/EqVupKaBM4GYwy+UISxQARvMJzRs/yJjg54eXhvD0+8ehzvhOsALmuaT/uqRq5evmDaLRFbW1tnHM0sIsyrdFDtdiW9GlgpVfrKvgD4QxFOB8KMBbLX6UeN4fm3kiRxtjSwfNGcrLx2nIgwt9JBdZVutq6Uiim7AhA/0/cFI3iDkVlFMEwnWRLnbZfHkjjnVTmTPMPsiQhzKhzUuLXjV0qdrSwKQDgSxRuK4M3AlbipSJTEufr8GtZPk8SZKXrGr5SaTskWAH/8LD8UIZDBNfqJRKKG5948yc493ezrGh4/7nbZuentS1h3eQNLF7iTPENmyITJXR3jV0olUzIFIH6W77eGdjJxJW4qBr1BfrKvl0cymMQ5U3MrnTrUo5RKWdEWgGg0NpbvD0UylreTjnwkcSbiqXBQ7XZS4dCgNqVU6oqqABgDg2PBnA3rTBYMxzZc2bW3mwO9o+PHs53EmcicCgfzteNXSs1QURWAYCTKoDc4/R0zLJ7E+di+Xga9ofHjF9R5WN/SyPtWLMppTPKcitg6fpdDh3qUUjNXVAUglxIlcdoE3nthHe2rGri0cX5OM3PcLgc1Hj3jV0plhhaASQKhCE//ro+de85N4rz1sno+cFkDdXMrctqmCqedBboZi1Iqw7QAWHqHfTyyt4eO/ccZ8Z/ZceXiJXNpX9XItRfV5XzIRTP5lVLZVNY9S7pJnLlitwnVbhfzKjWTXymVPXktACJyM/AvgB3Yboz5ci5edzZJnNlkE2FelZNqTehUSuVA3gqAiNiBfwVuALqA34rII8aY17L1mplI4swGTehUSuVDPr8BvBN40xjzFoCI/ABYB2S0AESihhcOTZ3Eef2KRbS3NGY9iTOZOZUOat0uvXpXKZVz+SwAjcCxCb93Ae+afCcR2QhsBGhoOi/lJx/1h3h8/3F2TZHEua6lgZvfnv0kzmSqXHZqPS5d0qmUypuCnwQ2xtwH3AdwacsV0wb8vNV/ml17e3jqtRP4JyVxtq9q5J3NtXkdZnE5bNR6XLhdBf+fXilV4vLZC3UDE0/pm6xjaQtHovzq4Cl27enm5QlJnFVOOze+fTHtLY05SeJMRpd0KqUKTT57o98CF4pIM7GO/w7gj9N5gngS56Mv99J/OjB+PB9JnIm4HDaq3drxK6UKT956JWNMWEQ2AU8QWwZ6vzHm1VQemyiJ893LF7C+JbdJnIlUOO3UuJ061KOUKlh57Z2MMT8Ffprq/Uf8If7rd1/id8fzn8SZiI7xK6WKRVH1UseH/YjV+V9Q56G9pZHrc5zEmYiO8Sulik3R9VbXXpSfJM5EbCJUu2NbMBZCe5RSKlVFVQAuWOjhix9Yme9mALGrd+dUOKj16NW7SqniVFQFoFCulvVUOKjRDVmUUkWuqApAvmkuv1KqlGgBSIHDZqN2jk7wKqVKi/ZoSdhEmF/lpNqtE7xKqdKjBSABTelUSpU6LQCTaEqnUqpcaAGw6BW8SqlyU/a9nV7Bq5QqV2Xb6zntNqrdTuZW5m9TGKWUyqeyKwDa8SulVEzZFACHzUa1x8ncCocu6VRKKcqgADhsNua7ncyr1I5fKaUmKtkCoB2/UkolV3IFwG4TqqtczKvSjl8ppZIpmQJgt8ViG+ZVOrFpPLNSSk2r6AtAPK9nfpV2/EoplY68BN2IyJdEpFtE9lp/bkn3OWwi1LhdLK11U+NxaeevlFJpyuc3gK8bY76a7oNEhHmVDqrduhOXUkrNRlENAdltwnk1VZrQqZRSGZDPnnSTiOwTkftFpCbRnURko4jsFpHdg6dOauevlFIZIsaY7DyxyFPAkilu+gLwG+AkYIC/B+qNMXdP95yrV682u3fvzmg7lVKq1InIi8aY1ZOPZ20IyBjzvlTuJyLfAh7LVjuUUkpNLV+rgOon/NoO7M9HO5RSqpzlaxL4KyLSQmwI6DDwsTy1QymlylZeCoAx5k/z8bpKKaXO0CU1SilVprQAKKVUmdICoJRSZUoLgFJKlamsXQiWDSLSDxzJdzuSWEjsArdSpO+tOOl7Kz7ZeF/nG2PqJh8sqgJQ6ERk91RX25UCfW/FSd9b8cnl+9IhIKWUKlNaAJRSqkxpAcis+/LdgCzS91ac9L0Vn5y9L50DUEqpMqXfAJRSqkxpAVBKqTKlBSADRORmEfm9iLwpIp/Nd3sySUQOi8grIrJXRIp6Nx5r97k+Edk/4VitiPxMRN6w/k64O10hS/DeviQi3dZnt1dEbslnG2dKRM4TkV+IyGsi8qqIfMo6XvSfXZL3lpPPTucAZklE7MDrwA1AF/Bb4E5jzGt5bViGiMhhYLUxpugvuBGRVuA08KAx5hLr2FeAAWPMl63iXWOM+dt8tnMmEry3LwGnjTFfzWfbZsvaP6TeGPOSiMwFXgTWAx+lyD+7JO/tdnLw2ek3gNl7J/CmMeYtY0wQ+AGwLs9tUlMwxnQCA5MOrwMesH5+gNg/vqKT4L2VBGNMrzHmJevnUeAA0EgJfHZJ3ltOaAGYvUbg2ITfu8jhB5gDBnhSRF4UkY35bkwWLDbG9Fo/HwcW57MxWbBJRPZZQ0RFN0QymYgsA1YBz1Nin92k9wY5+Oy0AKjpXGOMuQJoAz5hDTWUJBMbDy2lMdFvAMuBFqAX+FpeWzNLIjIHeAj4tDFmZOJtxf7ZTfHecvLZaQGYvW7gvAm/N1nHSoIxptv6uw/YSWzIq5SciO9Rbf3dl+f2ZIwx5oQxJmKMiQLfoog/OxFxEusgv2uM+bF1uCQ+u6neW64+Oy0As/db4EIRaRYRF3AH8Eie25QRIuKxJqYQEQ9wI7A/+aOKziPAXdbPdwEP57EtGRXvHC3tFOlnJyIC7AAOGGP+acJNRf/ZJXpvufrsdBVQBlhLtP4ZsAP3G2P+Ib8tygwRuYDYWT/E9o/+XjG/NxH5PrCGWNzuCeCLwC7gR8BSYlHjtxtjim4yNcF7W0NsCMEAh4GPTRgzLxoicg3wLPAKELUOf57YWHlRf3ZJ3tud5OCz0wKglFJlSoeAlFKqTGkBUEqpMqUFQCmlypQWAKWUKlNaAJRSqkxpAVBFT0QiVmLifhH5TxFxz+K5viMiH7Z+3i4iK5Pcd42IvGcGr3FYRBYmuK1FRIyI3Jzu8yqVLi0AqhT4jDEtVgpmEPj4xBtFxDGTJzXG3DNNqusaIO0CMI07geesv5XKKi0AqtQ8C/yBdXb+rIg8ArwmInYR2Soiv7UCtj4GsSsxRWSbtZ/DU8Ci+BOJyDMistr6+WYReUlEXhaRp63gro8Df2l9+3iviNSJyEPWa/xWRK62HrtARJ608t63AzJVw62rQv+QWMzxDSJSOeG2/2G18TkR+b6I/LV1fLmIPG6F9T0rIhdn/L+oKlkzOjNSqhBZZ/ptwOPWoSuAS4wxh6wk02FjzJUiUgH8SkSeJJa++DZgJbE0ydeA+yc9bx2xPJZW67lqjTEDIvLvTMhsF5HvAV83xjwnIkuBJ4AVxK7Kfc4Ys0VEbgU2JHgL7wEOGWMOisgzwK3AQyJyJfAh4HLACbxELDceYhuIf9wY84aIvAv4N2DtDP8TqjKjBUCVgioR2Wv9/CyxbJX3AC8YYw5Zx28ELouP7wPzgQuBVuD7xpgI0CMiP5/i+a8COuPPlSRu4H3AytiJPADzrJTHVuCD1mN/IiKDCR5/J7H9JLD+/gixkLCrgYeNMX7ALyKPwniC5HuA/5zwmhUJnlupc2gBUKXAZ4xpmXjA6hDHJh4CPmmMeWLS/TK51Z4NuMrqqCe3JSmJ7Sz3IWCdiHyBWHsXxMP4krze0OT3rlSqdA5AlYsngL+woncRkYushNNO4I+sOYJ64LopHvsboFVEmq3H1lrHR4GJHfSTwCfjv4hIi/VjJ/DH1rE2YKrNPa4H9hljzjPGLDPGnE/s7L8d+BXwARGptM763w9g5cYfEpE/tJ5bROTydP6jqPKmBUCVi+3ExvdfktjG6d8k9g14J/CGdduDwK8nP9AY0w9sBH4sIi8DP7RuehRoj08CA/8NWG1NMr/GmdVIf0esgLxKbCjo6BTtu5MzyatxDxHbX/q3xKKP9wEdxJIjh637/AmwwWrXq+h2pCoNmgaqVBEQkTnGmNPWNQ6dwMb4XrJKzZTOAShVHO6zLkqrBB7Qzl9lgn4DUEqpMqVzAEopVaa0ACilVJnSAqCUUmVKC4BSSpUpLQBKKVWm/j8acLwkWk37dgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# plot results\n", "sns.regplot(x=y_pred,y=y_test, scatter_kws=dict(color='k'))\n", "plt.xlabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### [Cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))\n", "\n", "Not perfect, but its not bad, as far as predicting with unseen data goes. Especially with a training sample of \"only\" 69 subjects.\n", "\n", "- But, can we do better?\n", "- On thing we could do is increase the size our training set while simultaneously reducing bias by instead using 10-fold **cross-validation**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\"logo\"
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Cross-validation is a technique used to protect against biases in a predictive model\n", "- particularly useful in cases where the amount of data may be limited. \n", "- basic idea: you partition the data in a fixed number of folds, run the analysis on each fold, and then average out the overall error estimate" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Let's look at the models performance across 10 folds" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# import modules needed for cross-validation\n", "from sklearn.model_selection import cross_val_predict, cross_val_score\n", "\n", "# predict\n", "y_pred = cross_val_predict(lin_svr, X_train, y_train, cv=10)\n", "# scores\n", "acc = cross_val_score(lin_svr, X_train, y_train, cv=10)\n", "mae = cross_val_score(lin_svr, X_train, y_train, cv=10, \n", " scoring='neg_mean_absolute_error')\n", "# negative MAE is simply the negative of the \n", "# MAE (by definition a positive quantity), \n", "# since MAE is an error metric, i.e. the lower the better, \n", "# negative MAE is the opposite" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fold 0 -- Acc = 0.419, MAE = 5.271\n", "Fold 1 -- Acc = 0.11, MAE = 3.069\n", "Fold 2 -- Acc = 0.79, MAE = 2.275\n", "Fold 3 -- Acc = 0.809, MAE = 3.061\n", "Fold 4 -- Acc = 0.641, MAE = 3.906\n", "Fold 5 -- Acc = 0.195, MAE = 4.732\n", "Fold 6 -- Acc = 0.684, MAE = 3.974\n", "Fold 7 -- Acc = 0.815, MAE = 2.693\n", "Fold 8 -- Acc = 0.058, MAE = 5.525\n", "Fold 9 -- Acc = 0.698, MAE = 2.571\n" ] } ], "source": [ "# print the results for each fold\n", "for i in range(10):\n", " print(\n", " 'Fold {} -- Acc = {}, MAE = {}'.format(i, np.round(acc[i], 3), np.round(-mae[i], 3))\n", " )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "For the visually oriented among us" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Metric score [i.e., R-squared 0 to 1]')" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAFzCAYAAAD18ZqMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABWCklEQVR4nO3dd3xUVfrH8c9JpSSEFloCJCT0FoqgIigSKVKVotJEXd3VddHdn72sus266rK6rroiHUSxAAIBCyoqSgsdEjpJKKEF0kg7vz/OBANCMgkzc+/MPO/Xa14h0+6XiHnmnnvOc5TWGiGEEELYS4DVAYQQQgjxa1KghRBCCBuSAi2EEELYkBRoIYQQwoakQAshhBA2JAVaCCGEsKEgqwOUVb9+fR0TE2N1DCGEEMIj1q1bd0xrHXmxx2xVoGNiYli7dq3VMYQQQgiPUErtv9RjMsQthBBC2JAUaCGEEMKGpEALIYQQNiQFWgghhLAhW00SE8JbFBYWkpaWRn5+vtVRRBmBgYHUrl2b+vXrExAg5x/Cu0mBFqIK0tLSCA8PJyYmBqWU1XEEoLWmsLCQI0eOkJaWRrNmzayOJMRlkY+YQlRBfn4+9erVk+JsI0opQkJCiIqKIicnx+o4Qlw2KdBCVJEUZ3uSoW3hK+RfshBCCGFDUqCFEF5l5cqVREdHWx1DCLeTAi2Ej4mJiaF69eqEhYXRqFEjJk2aRHZ2ttWxhBCVJAVaCB+0aNEisrOzSU5OZsOGDTz//PNWR7qk4uJiqyMIKxzZBiUlVqewNSnQQviwRo0aMWDAAJKTky/6+LRp02jRogXh4eHExsYye/ZswBTNhx56iPr169OiRQvefPNNlFIUFRUB5iz9iy++OPc+zz77LOPHjz/3/ejRo2nUqBERERH06dOHrVu3nnts0qRJ3Hvvvdx4443UrFmTr7/+moyMDEaOHElkZCSxsbFMmTLl3PPz8vKYNGkSderUoV27dqxZs8aVPyJhhUOb4K2rYPN8q5PYmqyDFuIyPbdoK9syTrv1GO2a1OKZoe0r/bq0tDSWLl3K9ddf/6vHcnJymDx5MmvWrKF169YcOnSIEydOAPDuu++yePFiNmzYQM2aNRk5cmSljjto0CCmTp1KSEgIjz76KOPGjTvvQ8KcOXNYsmQJixcvJj8/n969ezN8+HDmzp1LWloaiYmJtG7dmgEDBvDcc8+xe/dudu/eTU5ODoMGDar0z0HYzM4ljq9LofOt1maxMTmDFsIHjRgxgvDwcJo2bUqDBg147rnnLvq8gIAAtmzZQl5eHo0bN6Z9e/MhYP78+Tz44IM0bdqUunXr8vjjj1fq+HfeeSfh4eGEhoby7LPPsnHjRrKyss49Pnz4cHr16kVAQACbN28mMzOTP//5z4SEhNCiRQvuvvtu5s2bdy7Lk08+Sd26dWnatCmTJ0+u4k9F2EZKkvm652soLrI2i43JGbQQl6kqZ7bu9umnn5KYmMg333zD2LFjOXbsGI899hizZs0C4IknnuCJJ57ggw8+4JVXXuGuu+6iV69e/POf/6RNmzZkZGTQtGnTc+/XvHlzp49dXFzMk08+yYcffkhmZua5dcnHjh0jIiIC4Lz33r9/PxkZGdSuXfu89+jduzfAZWURNpR9FDLWQ6OOcHgzpK+DZj2tTmVLcgYthA+79tprmTRpEg899BD//e9/yc7OJjs7myeeeAKAAQMGsGLFCg4dOkSbNm24++67AWjcuDEHDx489z4HDhw4731r1qxJbm7uue8PHz587s9z5szhs88+44svviArK4t9+/YBphVnqbJNXpo2bUpsbCynTp06dztz5gxLlixxKovwMqkrzNf+fwcVALu+KP/5fkwKtBA+7sEHH2TFihVs3LjxvPuPHDnCZ599Rk5ODqGhoYSFhZ072x0zZgxTpkwhLS2NkydP8sILL5z32oSEBObNm0dhYSFr167lo48+OvfYmTNnCA0NpV69euTm5p77MHApPXr0IDw8nBdffJG8vDyKi4vZsmXLuclgY8aM4fnnn+fkyZOkpaXx73//2xU/FmGV1CQIbwyxfSD6CinQ5ZACLYSPi4yMZOLEifzlL3857/6SkhJeffVVmjRpQt26dfnmm2946623ALj77rsZMGAAnTt3pmvXrtx8883nvfavf/0ru3fvpk6dOjzzzDOMHTv23GMTJ06kefPmREVF0a5dO6688spy8wUGBrJ48WKSk5OJjY2lfv36/OY3vzl3zfqZZ56hefPmxMbG0r9/fyZMmOCKH4uwQlEB7P4aWvYHpSA+ETI2QM4xq5PZkio77GS17t2767Vr11odQ4gKbd++nbZt21odw2P27dtHbGwshYWFBAXZf+qKv/338Rp7voEZw+DWOdBmsLn+/O71cPP/oNNoq9NZQim1Tmvd/WKPyRm0EEIIz0hdDoEhEHut+b5xF6hRT4a5L0EKtBBCCM9ISYKYayA0zHwfEABx18PuL6Wr2EVIgRZCVCgmJgattVcMbwubOrEHjqdCywHn3x+fCDmZcHiTNblsTAq0EEII90tZbr626n/+/XGOLncyzP0rUqCFEEK4X2oS1G8FdVucf39YA2jcGXZ9aU0uG/PNAl1cCKteh22fWZ1ECCHE2WzYt8osr7qY+EQ4+BPknfJoLLvzzQIdEAQb58H3Uyp+rhBCCPfasxKKC6DVgIs/Hp8Iuhj2fuPRWHbnmwVaKegyDtLXQuZOq9MIIYR/S02C0FrQ7KqLPx59hXlcrkOfxzcLNEDHMaACIXm21UmEEMJ/aW36b8f1hcDgiz8nMBhaXGuuQ9uoeZbVfLdAhzc01zs2zpPtzIRfiYmJISQkhGPHzm+f2KVLF5RS5zavAHj22WdRSvHTTz+d99xp06YRGBhIWFjYebeMjAxP/BWELzm8Cc4c+vXyqgvFJ8LpdMjc4ZlcXsB3CzSYYe7sI7D7K6uTCOFRsbGxzJ0799z3mzdvPm/3KTC7S82YMYO6desyY8aMX73HVVdddW73q9JbkyZN3J5d+JjS5VUtbyj/efGJ5qsMc5/j2wW65QDTRi55ltVJhPCoCRMmnFd0p0+fzsSJE897znfffcehQ4eYMmUK8+bNo6CgwNMxhT9ITYKobmY5VXkioiGyrRToMny7LVBQiLkWvfY9yD0BNepanUj4oqWPmY3n3alRRxj0QsXPc7jyyiuZOXMm27dvp1WrVsybN4/vv/+ep5566txzpk+fztChQxkzZgyTJ09m0aJFjBw50h3phb/KOQZpa+G6x517fnw/+PkdKMiBkJruzeYFfPsMGswwd3EBbP6o4ucK4UNKz6JXrFhB27ZtiYqKOvdYbm4uH374IWPHjiU4OJhRo0b9aph79erV1K5d+9wtLi7O038F4e1SVwD6193DLiU+0fy+3rfKrbG8hW+fQYM582jU0Qxz97zH6jTCF1XizNaTJkyYQJ8+fdi7d++vhrc/+eQTgoKCuPHGGwEYN24ciYmJZGZmEhkZCZiz8FWr5BeluAypSRDWEBp1du75za6C4BpmmPtSa6b9iO+fQQMkjIdDG+HwFquTCOExzZs3JzY2liVLlnDzzTef99j06dPJzs6mWbNmNGrUiNGjR1NYWMicOXMsSit8TnEh7PrKTA4LcLLUBFeDmN5yHdrBPwp0x9EQEAzJ8stH+Jf33nuPr776ipo1f7mel56ezpdffsnixYtJTk4mOTmZjRs38uijj150NrcQVXLwJzibVfHyqgvFJzp2vtrtnlxexK0FWim1Tym1WSmVrJRa685jlatmPWg9EDZ9YD7VCeEn4uLi6N69+3n3fffddyQkJNC/f38aNWp07jZ58mQ2bdrEli1mpOnHH3/81TroNWvWWPHXEN4oJcmcGMX1rdzr4vuZr7I81iPXoPtqrY9V/DQ3SxgP2xdB6nJoM9jqNEK4TdlGJGUFBQWhHV2aHnvssV893qRJEwoLzQfYDh06MGnSJHdFFP4gJQliekFoeOVeVy8O6sSaYe4ed7snm5fwjyFuMMMmNRvABmn9KYQQbnVyHxzbWfnh7VLxibD3Wyg669JY3sbdBVoDy5VS65RS1k6hDgyCzreYWYXZmZZGEUIIn1baPayqM7HjE6EwFw786LpMXsjdBfoarXVXYBDwe6VUnwufoJS6Rym1Vim1NjPTzYUzYRyUFMHm+e49jhBC+LPUJKgbZ4arqyLmGggMcayj9l9uLdBa63TH16PAJ0CPizznHa11d61199L1l27ToC006WqGuWXHFCGEcL2CHNj73eWtYw4NM2uid33pulxeyG0FWilVUykVXvpnoD9g/ULkLuPg6FazLlqIy6DlQ54tlZSUWB3Bv+39ForPXn6jkfhEyNwOWWmuyeWF3HkG3RBYpZTaCPwMfK61XubG4zmnw0gIDJV9osVlqVatGsePH5cibSNaawoKCkhPTz9v3bfwsJRlEBIOza6+vPc5t7uV/55Fu22ZldZ6D+BkfzcPql7HLLPa/CH0/xsEhVqdSHih6Oho0tLScPu8CVEpQUFBREREUL9+fauj+CetzXXjuOvMZkWXo0FbCG9illt1u90l8byN7/fivpgu42Drx7BzKbQfYXUa4YWCg4OJjY21OoYQ9nJkC5xOd373qvIoZZqWbPvMNJgKDL789/Qy/rMOuqwWfc0nMxnmFkII10lJMl9bOrl7VUVa3gBnT5stK/2QfxbogEDofKsZOjlz2Oo0QgjhG1KXQ+MECG/omveLvRZUoN9unuGfBRrMmmhdAhvnWZ1ECCG8X85xSFvj2m0iq9eGpj2kQPud+vHQtKcZ5paZuEIIcXl2fWFOely9j3N8PziU7JcdIP23QIM5iz6WAunrrE4ihBDeLTXJ7HfQuItr37d0uZUf7m7l3wW6/U0QVB02zLI6iRBCeK/iInMG3fIGCHBxWWnUGWrU98thbv8u0NVqQbthsOVjKMyzOo0QQnintJ8hP8t1s7fLCggww9y7vwQ/6xLn3wUazDD32SzY8bnVSYTwLUVn4chWq1MIT0hJgoAgiOvrnvePT4Tc4+ZatB+RAh3TGyKayTC3EK5UkAOzRsJbV8O3L8tETF+XutxsblEtwj3vH3c9oPyu7acU6IAASLgN9qz066bsQrhMQQ7MHgP7v4fm18BXf4Olj/rd8KTfOHUAjm6DVgPdd4ya9aFJgt9dh5YCDdD5NkDDxrlWJxHCu53Nhtmj4cAPcPO7cPsiuOp++Plt+PhuKCqwOqFwtdLuYa5eXnWh+ERzrTvvpHuPYyNSoAHqxppP+slzZChOiKo6V5xXw8j/QcdRZoSq/98g8TnY8hHMvcU8T/iO1OVQJxbqxbv3OPGJZp31npXuPY6NSIEu1WUcnNhjfrkIISrn7BmYPQoO/mSKc4eRvzymFFzzIAx/E/Z8AzOGma5TwvsV5Jr9n1sNMP+d3SmqO4RG+NUwtxToUm2HQXBNSJbJYkJUSv5pMyHs4M8w6j3ocPPFn9dlPNwyy8zsnjrAXLsU3m3fd1CU757lVRcKDDLbWO760m9GOqVAlwoNM41Ltn5qJrkIISpWWpzT18Ho983/Q+VpcyNM+ASyj8J7A+Dods/kFO6RkmRObGKu8czx4hPhzCEzKc0PSIEuq8s4KMiGbQutTiKE/eVnwaybIWM9jHof2g137nXNr4Y7l5rriVMHwoGf3JtTuIfW5vpzi+sgKNQzx4zrZ776yTC3FOiyml1lJjvIPtFClC8/C2beDBkbYPQ005GvMhq2h7uWQ416MGP4LzOBhfc4ug2yDrp/9nZZEVHQoL0UaL+klOkstu87OLnP6jRC2FPeKZh5ExzaCGNmQNuhVXufOs3hziSIbA1zb4NkWeboVUo/VHni+nNZ8f1g/49+sRpACvSFOt8KKPllIcTFnCvOm0xxbjP48t4vLBImLYbY3vDp7+D7KS6JKTwgdTk06gS1Gnv2uPGJUFJoTqR8nBToC9VuCi2uNWuipfOREL/IOwkzR8DhzXDLTDPhyxVCw2HsfGh/M6x4GpY/7TezdL1W7gmzpM6Tw9ulml1pJqb5wTC3FOiLSRgPWQdg/yqrkwhhD3knYcYIs0TqllnQepBr3z8o1KyfvuJu+GEKfHofFBe69hjCdXZ/ZSb5tbSgQAeFQmwfSF3h8x/kpEBfTJvBEFoLNshkMSHIPWEmch3d5ijObuq5HBAIN74M1z0BG+fAvHGmEYawn5QkM8Evqqs1x4/vB6f2m+ZSPkwK9MWE1DDNFrZ9ZtZ5CuGvzhXnHXDrHPcPaSoF1z0Kg1811zhnjjAZhH2UFMOuFWZyWECgNRniE81XHx/mlgJ9KQnjoSgPtn1qdRIhrJF7wrTlzNxpinPLGzx37CvugjHTzTKu92+ErHTPHVuUL22NueTh6dnbZdWNhbpxUqD9VnR3qNdShrmFf8o5DtOHQWYK3DYHWiZ6PkO74TB+gdkGduoAk0VYLyUJVKBjj2YLxSfC3u+gMN/aHG4kBfpSlDKdxQ6uhuO7rU4jhOfkHDdnzsdT4ba5vwwnWiG2D9zxuen3PHUApK2zLoswUpebpk7Va1ubIz7RjHIe+MHaHG4kBbo8nW4FFSCdxYT/yDkG04fC8V1w2zwzGcdqjTubhibVaplsu760OpH/ykqDI1uglYXD26ViekFgKKT67jB30KUeUEr9yYnX52it33ZhHnup1dj0fk2eC32ftG5ChBCekJ1pzpxP7IWxH5gey3ZRL84U6VmjYM4tcNN/zX7TwrNSl5uvViyvulBITdPXfdcXwD+sTuMW5Z1BPwyEAeHl3P7P3QEt12UcnMnwq03ChR/KzjRnp3YszqXCG5nh7qY9YMFdsPq/VifyPylJULu5ac9qB/GJcGynz25deskzaGCm1vov5b1YKVXTxXnsp9UgqFbbDHPbYbhPCFfLPmqK88n9MG6+ue5rV9UiYPzHpkAvexRyMuH6p8ycEeFehXmw5xvoOsE+P+/4RFj+pLns0f0Oq9O43CXPoLXWj1T0Ymee4/WCq0HH0bB9selDLIQvOXMEpg0xZyDjPrR3cS4VXA1GT4euE+G7V2DRA1BcZHUq37dvlZmUZYfh7VKRraFWtM8ut6rSJDGllO99VClPl3FQfBa2LLA6iRCuc+YwTB9iJv6M+8hsWOEtAoNg6BTo/RCsnw4f3u7Ty21sISUJgmtAzDVWJ/mFUmZkc883PtkatqqzuJ9zaQq7a5wADdrJbG7hO84cNmfOWekw/iMzI9bbKAX9noZBL8GOxTBrpNmnWrie1pCaBLHXmhEMO2l5AxScgYM/W53E5S5ZoJVSmy5x2ww09GBG65XuE52+zrQ8FMKbnT4E0wbDmUOmEUjzq61OdHl6/hZGvmd2V3p/sBm2F66V6ZiIZYflVReK7QMBQT45zF3eGXRDYCIw9CK34+6PZjOdxpjuOXIWLbzZ6QxHcT7sKM5XWZ3INTqOMrPPT+yBqf19fhMFj0tZZr5a2d7zUqpFQNOeflegFwNhWuv9F9z2ASs9ks5OwhqYjQI2fSATUoR3yko3xTn7qJkJ3exKqxO5Vnw/uH2R2eDmvf5waKPViXxH6nJo2BEioq1OcnHx/eDwJp8bPSlvFvddWuuLboistR7rvkg2ljAOso/AbulkJLxMVpqjOGfChI+hWU+rE7lHdDfT0CSomhnu3vut1Ym8X95JOLDansPbpUrb0e7+ytocLiatPiujZX+zB+qGWVYnEcJ5pcU59zhM+MQ0+vBlka1MkY6INhPHtn1mdSLvtvsr0MX2Wl51oYYdoWYDnxvmlgJdGUEh0OkW2LlU9qgV3uHUQUdxPuEozldYncgzIqLgjiXQpAvMvx3WTrU6kfdKWQ7V65od/uwqIMAMc+/+yuxX7SOkQFdWwjgoKYTNH1qdRIjynTrgKM4nYcKn9v4F6w416pq/d8v+sPiPsPJFs1xIOK+kGHatMEPIdt+LID4R8k5ARrLVSVzGqQKtlGqolBriuDVwdyhba9QBGnWSYW5hbyf3m+KcdwomfmKuzfqjkBpw62zoPBZW/gOWPAwlJVan8h7p68ylkVY2Ht4u1aIvoHxqmLvCAq2UGgP8DIwGxgA/KaX8exuZLuPNjMHDm61OIsSvndxvmpDkZ8HETyHKT4tzqcBgGPEfuPoPsOZd08e76KzVqbxDSpJZXhp3vdVJKlazHkR19a8CDTwJXKG1vl1rPRHoATzt3lg212EUBARD8hyrkwhxvpP7zJnz2dMwcaH5hSVMs6H+f4Mb/gJbP4Y5Y+DsGatT2V9qklljXKOu1UmcE58I6Wt9Zo6QMwU6QGt9tMz3x518ne+qWQ9aDzJroosKrE4jhHFirzlzPnsGJn4GTRKsTmQ/vR6AEW/B3u/MDl45x6xOZF+nM8wooZ2XV10oPhF0Cez52uokLuFMoV2mlEpSSk1SSk0CPgeWujeWF+gy3lybKd3AXAgrndhjinNBNty+UIpzeRLGwq1zTNve9/qbSwLi10p/t9l5edWFmnQ12wPv8o1eFRUWaK31w8DbQCfH7Z3KbDOplApUSm1QSi2uekwbiusHYQ2l9aew3vHdpjgX5ppOWo07W53I/loPNKMMucdMkT6y1epE9pOyHCKaQoO2VidxXmAQxPU116F9YMa+M5PEXtRaf6y1/pPj9olS6sVKHOMBYHvVI9pUYJBZE52SZFonCmGFc8U5z5w5N+podSLv0awn3LHMXJ9+fxDs/9HqRPZRmA97VpolakpZnaZy4hNNx8cjW6xOctmcGeK+4SL3DXLmzZVS0cBg4H+VCeU1EsaZDjub5ludRPij47vNhLDis+bMWYpz5TVsB3ctN12oZo4wTYgE7F8FhTnesbzqQnH9zFcfmM1d3naT9zq2lmx9wXaTe4FNTr7/68AjwCUXHiql7lFKrVVKrc3MzKxMdus1aGOWsCTP9onhFOFFju2C9280m9TfvtiszxdVU7uZaQ3aoB3MGyc9DsAMbwdVN1s5eptajU3rTx+4Dl3eGfQczNaSCzl/q8luWuvxFb2xUmoIcFRrva6852mt39Fad9dad4+MjHQ+uV0kjIOj2+BQstVJhL84lmrOnEuKzJlzw3ZWJ/J+NeuZn2VsH/js97DbN2YBV4nWZnlVbB8Irm51mqqJ7wcHfvT6pXTl7WaVpbXep7W+7YLtJp1dYNYLGKaU2gfMA65XSvneR9MON0NgKGyQyWLCAzJTTHHWxTBpsRRnVwoNg9vmQb14WDQZCnKsTmSNY6lmPb03La+6UHyi+QDr5buZuW09s9b6ca11tNY6BrgV+MqZM2+vU70OtB1ienNLdyLhTpk7HcVZm2Ftb5pd6y2Cq8GwN0wf86/+ZnUaa6Qmma/etLzqQk17QkiY11+H9u+GI66SMA7yT8HOJVYnEb4qc6eZrQ3mzLlBG2vz+LLmV8EVd8Pqt+DgGqvTeF6K43p87aZWJ6m6oBCIvdbrl1s5s8yqoVKqq+PWsCoH0Vqv1FoPqcprvUKL66BWlAxzC/fITDHFWSmY9DlEtrY6ke9LfMbsJ73wfv8aGcvPMtduW3rx8Hap+H5mJOT4LquTVFl5s7gTlFKrgZXAS47bN0qp1UopafBbVkAgdL4Vdn8Jpw9ZnUb4kuO7TUtKMMPaka2szeMvQsNhyOuQuQO++6fVaTxn91fm2m2rgVYnuXzx3r/cqrwz6GnAA1rrtlrrRMetDfAg8L4nwnmVhHGmB+ymeVYnEb6itH1nSZFpQiLF2bNaJkLn20yBPuz9TS+ckrLctMqMvsLqJJevTgzUa+mzBbqm1vqnC+/UWq8GarovkpeqFwdNrzTD3F58zUPYxMl9MG0oFOWblpQyIcwaA/5hJoIuvB+Ki6xO414lJbBrhZkBHRhkdRrXiE+EfatMpz0vVF6BXqqU+lwpdYtS6mrH7Ral1OfAMk8F9CpdxsHxVEhba3US4c1OHTDFuSDbFGdpQmKdGnXhxpchYwOs/o/VadwrYwPkZHpn97BLiU80H3L3f291kiopbx30ZOANoC/wuOPWF3hTa32/Z+J5mXYjTPedZN9b7i08JCvNsWVklinOjTtZnUi0GwFthsDXfzdzAnxVahKoAFPUfEVMLwiqBqneOcxd7ixurfVSrfXvtNZDHbffaa1lLdGlVKsF7YbDlo+9dkhFWOh0hinOeSdhwieyZaRdKAU3vmIaEi2cbIaCfVHKMnPtuUZdq5O4TnB1aN7La69DyzpoV+syDs6ehu2+tbumcLMzh01xzjkG4z82Pd6FfdRqDAP+ZjaRWD/d6jSud+YwHNroG8urLhSfaC49ntxndZJKkwLtas2vMc33ZZhbOCv7qFlKlX0Exi+Apj4wg9YXdZlg+lOv+DNkpVudxrVSl5uvvrC86kKlQ/ZeuHmGFGhXCwiAzmNhzzdw6qDVaYTdZWea4pyVBuM+NHsUC3tSCoZOMcvePv+Tb63WSEkyzZYatrc6ievVbwkRzbyyQF9yLr1S6t/AJf8FOiaRiYtJuA2+eQE2zoNrH7Y6jbCrnOMwYzic3G+Kc/OrrU4kKlI3Fq5/GpIehy0LoOMoqxNdvqKzsGcldBxtPoT4GqVM05LNH0JRgWkD6iXKO4NeC6wDqgFdgVTHLQHwnr+hFerEQExv2SdaXFruCVOcT+yGsfMgtrfViYSzev4WorrD0kfMnAFvt/8Hs6TPl5ZXXajlDebvePBXrT1srbxlVtO11tOBTsB1Wut/a63/DfTDFGlRnoRxcHKv6WsrRFl5J2HmCDiWArfOMb3chfcICIThb0D+aVj2mNVpLl9KkpmhHtvH6iTuE9sHAoK8bja3M9eg6wC1ynwf5rhPlKfdMLPdmWygIcrKOwUzb4Kj2+GWWb/0CxbepUFb6POwGTbd6eV9m1KTzAhOiA83iAwNh2ZXed11aGcK9AvABqXUNKXUdGA98A/3xvIBITWh/QjY+gmczbY6jbCD/NMwa6Tp6zxmBrTywSUt/uSaP5ptGRf/0fy39UbHdpme7744e/tC8f3gyGav2tCowgKttX4f6Al8AnwMXOUY+hYVSRgPhTmwfaHVSYTVzp6B2aPgUDKMngatB1mdSFyuoBAY9gZkH4YvnrE6TdWkJpmvvrj++UKly612f2VtjkpwZj9oBSQCnbXWnwEhSqkebk/mC5pdCXVbyDC3vyvIgdljTI/2UVOhre9uje53orvBlffB2qlmUwZvk5IEkW2gTnOrk7hfww4Q1tCrrkM7M8T9H+Aq4DbH92eAN92WyJcoBQljTfehE3utTiOsUJALc26Bg6th5P9MK1jhW/o+aVZuLPyDd7X4zT9tZnD7w9kzOJZbJTr2vC62Oo1TnCnQPbXWvwfyAbTWJ5FlVs7rfBugYONcq5MITyvMg7m3mp10bnoHOtxsdSLhDiE1YNi/zbXclc9bncZ5e76GkkLfXl51ofh+kH8K0tdbncQpzhToQqVUII6mJUqpSMBHu8W7QUS0WUaTPNd3m+yLXyvMh3ljYe+3MPw/0Gm01YmEO8X2gW6T4Id/e80vf1KWQ2gENPWj7nUt+podu7xkmNuZAj0FM0GsgVLq78AqZBZ35XQZD1kHYN93VicRnlB0Fj4Yb4bShr9hOssJ33fDX8w1zs/uNx2r7KykxPTfju8HgcFWp/GcGnXNRjS+UKCVUgHAXuAR4HngEDBCa/2hB7L5jjaDzSfVZJks5vOKCmD+RNi1Aob+y3w4E/6hWgQMeQ2OboXv/2V1mvIdSoaco/41vF0qPhHS15lufjZX0X7QJcCbWusdWus3tdZvaK23eyib7wiubq4/blvoveslRcWKC+HDSWZf3cGvmiFP4V9aD4IOI+Hbl+DoDqvTXFrqckD9svTIn8QnAtorlls5M8T9pVJqpGO5laiqLuOhKM80LhG+p7gQProTdn4Og16GK+6yOpGwysAXTRfBhX+w72zhlCSI7g4161udxPOadIHqdbximNuZAv1b4EPgrFLqtFLqjFJKTgMrK6ob1G8lw9y+qLgIPr7HNKQZ8Dz0vMfqRMJKYZEw6EVI+xl+ftfqNL+WfRQy1kNLPxzeBtNLPe560/bT5hN3nekkFq61DtBah2itazm+r1XR68QFlDIbaBz8ybTXE76hpBg+/R1s/Rj6/w2uus/qRMIOOo4264u/fM5sJ2onqcvNV39uNRufaK7BH9lsdZJyOXMGjVKqjlKqh1KqT+nN3cF8UqdbzBR/OYv2DSXF8Ol9ZsOEfs/A1X+wOpGwC6XMhDEVAIsesNe2sylJEN4YGnWyOol14hyb1Nh8mNuZVp+/Ab4FkoDnHF+fdW8sH1WrsfnktnGefa9NCeeUlMDCybBpHvR9Cnr/yepEwm4iouGG50xDELt8KC8qgN1fm7N7f55WFN7QfECx+e5WzpxBPwBcAezXWvcFugCn3BnKpyWMgzMZ5n9a4Z1KSmDxA5A8C659DK592OpEwq663QnNe0HSE3DmsNVpzP70BWf8c3nVheITzSXH/Cyrk1ySMwU6X2udD6CUCtVa7wBauzeWD2s9yMwglA00vJPWsOT/YP0M6P0QXPeY1YmEnQUEwNAppnnNkoesTmOuPweGQOy1ViexXnwilBSZbn825UyBTlNK1QY+BVYopT4DbDbrwYsEhZoJJDs+h7yTVqcRlaE1LH3E7FzU60G4/in/HiYUzqkfD9c9DtsXwbbPrM2SkgQx10BomLU57KBpDwgJt/V1aGdmcd+ktT6ltX4WeBp4Dxjh5ly+LWEcFJ+FLQusTiKcpbUZpvz5Hbjqfkh8VoqzcN5V90PjzvD5Q9Z1sDq+G46n+u/yqgsFBkOLa811aDtN4ivDmUlizUpvmLafyUAjdwfzaY07Q4P2MsztLbSGFU/D6v9Az3vNciopzqIyAoNg+JuQdwKWP2VNBlle9WvxiZB1EI6lWJ3kopwZ4v4cWOz4+iWwB1jqzlA+TynoMs40CzgqnVNtTWuzlvWHf8MVd8PA56U4i6pp1NFcGkmebc3s4ZQk0yypbgvPH9uu4u293MqZIe6OWutOjq8tgR7Aj+6P5uM6joGAIPssvxAX9/U/YNVr0O0OuPFlKc7i8vR52BTJRQ/C2WzPHfdsttmXvKWcPZ+ndjOo39p7C/SFtNbrAT/aQNRNwiLNtaCNH5g+zsJ+Vr5oNj3oMsFsfiHFWVyu4Gow7A0zrPrlXzx33D0robhAllddTHwi7PseCnKtTvIrzlyD/lOZ20NKqTlAhgey+b4u40y7OZsvlvdL374MK/9hJvQNnWKWywjhCs16Qs/fmgmHB1Z75pipSRBaC5pd5ZnjeZP4fmbS7v7vrU7yK8781gkvcwvFXIse7s5QfqNlf6hR3zS8EPax6jX46m+mNeuwf0txFq53/dMQ0dTseFWY795jaQ0pyyGur5m5LM7XvBcEVbflMLcz16CfK3P7u9Z6dmnjEnGZAoNNEdi5DHKOW51GgJkM9sWz0GEUjHjL7HwjhKuFhsHQ183s4W9fdu+xDm2E7MOyvOpSgquZteGpK6xO8ivODHEvUkotvNTNEyF9WsJYKCk0Gy4Ia61+yyyBaTcCbnpbirNwr/h+5hLK96/DoU3uO07p8qqWN7jvGN4uPhFO7IYTe6xOch5nxu72AHnAu45bNrAb+KfjJi5How5mXbQMc1vr53dh2WPQdiiM/J9ZtyqEu/X/G1SvCwvvN/uKu0NKktmPPqyBe97fF8Qnmq82mw/kTIHupbW+RWu9yHEbC/TWWn+jtf7G3QH9QsJ4OLzZvZ+ixaWtnWr6JLceDCOnynU64Tk16sLgV8ww9I9vuP79c45B+joZ3q5IvTio3dwrC3RNpdS5le1KqVigpvsi+aGOo0wD++Q5VifxP+tnwOI/ml9go9+HoBCrEwl/0264GblZ+Twc2+Xa905dAWjpHlYRpcxZ9N5vzcYmNuFMgf4jsFIptVIp9Q3wNWYLSuEqNeqaXa42zzf7tQrP2DDb7OkcnwhjZpiNTISwwo2vmH9/C/9gtjN1lZRlENYQGnV23Xv6qpY3QGGO55a+OcGZWdzLgJaYojwZaK21Xu7uYH4nYTzkHjfrFYX7bfwAPvs9tLgObpllZnIKYZXwRjDgH3DgB1g31TXvWVwIu78yhUeWClYspjcEBNtquZUzs7hHAyFa643AUGCuUqqr25P5m7jrIawRrHzBut1u/MWmD+HT30Fsb7h1DgRXtzqREGZGd4u+sOIZOHXw8t/vwGo4e1quPzsrNAyaX2Wr69DOfKx6Wmt9Ril1DdAPs93kWxW9SClVTSn1s1Jqo1Jqq1LqucsN69MCg2D4G3AsFaYNhjNHrE7km9a+Dx/fbZoT3DYPQmpYnUgIQykY+i/TWGTxHy9/C8TUJHNGGNfXNfn8QXwiHN0Kp+3RLNOZAl3s+DoYeFdr/TngzEyas8D1WuvOQAIwUCl1ZZVS+ouWN8C4+XByH7w/CLLSrE7kW374Nyx+0PFz/hBCZK6jsJk6zaHfn2HXisvvjZCyHGJ6QWi4a7L5A5stt3KmQKcrpd4GbgGWKKVCnXmdNkq3awl23Oy5K7adtLgOJnwCOZkwdZDtFs57Ja3h6+d/aUJyy2wZ1hb21eNuiO4BSx+F7MyqvceJvXBspwxvV1aDdhDe2DbXoZ0p0GOAJGCA1voUUBd42Jk3V0oFKqWSgaPACq31Txd5zj1KqbVKqbWZmVX8x+hrml0Jty+EgjOmSB/dYXUi76U1JD0J37xgJuKNmipLqYS9BQSaHvAF2bDs0aq9R2n3MNm9qnKUMh3e9nztvsYxleDMmXCu1vpjrXWqUuoerfUhZ2dxa62LtdYJQDTQQynV4SLPeUdr3V1r3T0yMrLSfwGf1aQLTFoCugSm3ShNTKqipBgWTYbVb0LP3zk2vpD2ncILNGgDfR6BLQtgx5LKvz4lCerGmQYconLiEyE/yzR4sVhl597/rioHcZx5fw0MrMrr/VbDdnDnMrPTyvQhcHCN1Ym8R3EhLPiNaUTS52EY+IIsNRHepdcD0LADfP4nyDvl/OsKcmDfKjl7rqoW14EKsMUwd2V/Yzm9Y71SKlIpVdvx5+rADYCM1VZWvTi4c6np1ztzBOz9zupE9leYBx+Mh60fQ+JzcP1TZuhKCG8SFGJGfbKPwIo/O/+6Pd+Y/Y2lQFdN9ToQfYVXFuihAEqpXk48tzHwtVJqE7AGcw16cSWPJwBqN4M7lkKtKJg9ClKt/4djW2ezYfZoM8Q3+FW45kGrEwlRdVFd4ar7Yf10U3idkZoEIeHQ7Gr3ZvNl8YmQscH0MrfQJQu0Y4LXbUqph8pcO05QSv0AVNjVXWu9SWvdRWvdSWvdQWv9F1eF9ku1GsMdS6B+S5h7K2xfZHUi+8k7aUYZ9v8AN78DV9xldSIhLl/fJ6BuCzOfoiC3/Odqbfpvx10nkyEvR3w/QJtObBYq7wz6PeA3QD1gilJqFvAK8JLWuosnwokL1KwPty+GJgkw/3bYNN/qRPaRfRSmDTG7Ao2ZAZ3GWJ1ICNcIrm6Guk/ug6//Xv5zj2yB0+myvOpyNe4CNepZPsxd3qa33YFOWusSpVQ14DAQp7U+7plo4qKq1zbrpOfeBh/fA4W50G2S1amsdeqgOXM+nQFjPzBtU4XwJTHXQPc7YfV/oP3NEN3t4s9LWWa+tpTdqy5LQID5PbLrS7N5iUUTTMs7aoHWugRAa50P7JHibBOh4aYTVnwiLHoAfvyP1Ymsc3y36bqWfdR8cJHiLHxV4nOmicbC+y+9613KcmicAOENPRrNJ8UnQu4xOLzRsgjlFeg2SqlNjtvmMt9vdkz8ElYKrg63zjb7yCY9Dt++bHUizzuyzRTnwly4fZFp8CKEr6pWC4a8Bke3warXfv14znFIWyOzt12l9MO+hcPc5Q1xt/VYClE1QaEwahp8dh989TczgaTfn/1jSVH6Opg1EoKqmRnuka2tTiSE+7UaAB1Hmw/kbYeaXgmldn0BaCnQrhLWwIxG7PrS9FKwwCXPoLXW+8u7eTKkKEdgEIz4r7kOvepVWPaYazd8t6N9q2D6MAitZRq5SHEW/mTgC+ZseuH9plteqdQkqNnATHASrhGfCAd/rlyjGBeS1kq+ICAAhrwOV94HP/3XLMco+z+uL0ldYc6ca0WZ4lwnxupEQnhWzfow6CUzivTTf819xUXmDLrlDdIxz5XiE0EXw14n16C7mPyX9BVKwYB/mKGYDTPNDO/iQqtTudZWx+z1+q3MmvBaTaxOJIQ1OoyEVgPhy7+anasO/mT6R8vsbdeKvgJCIyy7Di0F2pcoZdpa9nsGtnxk1koXnbU6lWtsmA0f3QlR3WDSYnMWIYS/Usp0ygsMNiNmKcsgIAji+lqdzLcEBkGLa811aO353ZIrXaCVUtOVUm9dbGcqYRO9/2SGwHZ+brqOVdR9yO5+ettMhGtxHUz4GKpFWJ1ICOtFRMENf4G938LP70Czq+T/DXeITzTNXzI9v5VEVc6g3wC+ACa4OItwpZ6/hWFvwJ6V5ppt/mmrE1We1vDtK7D0EWgzBG6bByE1rU4lhH10vR1iekNRvhnyFq4X3898tWCYu9IFWmu9Rmu9QGtdxZ3Ehcd0nQAj/wdpP5tOW7knrE7kPK3hi2fhq79Cp1tg9HSzrEwI8YuAANMGtOUAc11auF5ENES29Y4CDaCUusfVQYSbdBgJY2bC4c0wfShkZ1qdqGIlJfD5/8H3r0P3u8wyssDyluwL4cfqxsK4+WZDHeEe8f3MJjwFOR49bFUniflBJwwf0uZG06O6tC3m6QyrE11acRF8ei+sfc9sWD/4n7JsRAhhrfhEKC4wPRg8qEq/+bTWb7s6iHCzuOvNBKszh2HqQLMzjt0UnYUPb4dN8+D6p03vYX/oiiaEsLdmV0FwDY8Pc1dYoJVSDZVS7ymlljq+b6eUko12vVHzq+H2z8x6yamD4Fiq1Yl+UZBjZpzvWGxmoPd5SIqzEMIegquZyXipKzx6WGfOoKcBSUBpV4gU4EE35RHuFtUNJn0OJYVmuPvwFqsTmQ8MM282M86Hv2lmoAshhJ3EJ8LJveZSoYc4U6Dra63nA6VbTxYBPtpH0k806mA2mAgIhmmDTctAq+QcM5PX0tfBqKnQZbx1WYQQ4lLOLbf60mOHdKZA5yil6gEaQCl1JZDl1lTC/eq3hDuXmsYG04ebGYqedvoQvH8jZO6E2+ZC+5s8n0EIIZxRL85cfvPgnvPOFOg/AQuBOKXU98AM4A9uTSU8o06M2XCiVmMzxLz7K88d++Q+eH+gmVE+foFp8i+EEHbW87dQP95jh6uwQGut1wPXAlcDvwXaa603uTuY8JBaTWDSEvPpcM4tsGOJ+4+ZudPMJM87ZSatxVzj/mMKIYSXcWqZlda6SGu9VWu9RWtdqJRq5O5gwoPCIuH2RdCoI3wwHjZ/5L5jHdpoJqeVFJsdqaK6ue9YQgjhxaraAeI9l6YQ1qtRFyZ8Cs2uhAW/gfUzXX+MA6th2lCznvDOZdCwveuPIYQQPqKqjUoGuzqIsIFqtWDcR2bLuoX3w0/vuO69d38FM28yZ+t3LjND6kIIIS7pkgVaKVXL8bXuxW6eiyg8KqSG2TWq9WBY+jCseu3y33PH5+b6dt0WZnlXRPTlv6cQQvi48nYgmAMMAdZhlliVbeukgRZuzCWsFBQKY6bDJ78zO0oV5EDfJ6vW2WvTfPM+TbrA+I+geh2XxxVCCF90yQKttR7i+BrruTjCNgKD4eZ3ILg6fPsyFOTCgL9XrkivnQqL/2Rmad82F0LD3ZdXCCF8zCULtFKqkdb6cHkvduY5wosFBMLQKRBSE1a/CYU5MPg153aX+v5fsOLPZp/aMdNNoRdCCOG08oa4lwBdK3i9M88R3iwgAAa+YGZer3oVCvNg+H8uvT+z1vD1P+Dbl6D9zeYsPDDYs5mFEMIHlFegOyulTpfzuALKe1z4CqUg8RlzJv3VX6EwF0ZOhaCQ859XUgJJT8BPb0HXiTDkdXMWLoQQotLKuwYtv1nF+fo8ZM6kkx6HeWPhlpm/DF2XFMOiybBhFlz5+8pfrxZCCHGeqjYqEf7qqvtg6L/MxuWzR8PZM1BUAAvuMsX52sekOAshhAuUN8QtxMV1m2TOpD/5nWk+Ur0OpC6H/n+Dq2UfFSGEcAUp0KJqOo0xw9sf3gElReZ6c/c7rE4lhBA+w6kCrZS6BmiptX5fKRUJhGmt97o3mrC9tkNNZ7CCbNMeVAghhMtUWKCVUs8A3YHWwPtAMDAL6OXeaMIrNL3C6gRCCOGTnJkkdhMwDMgB0FpnANISSgghhHAjZwp0gdZaY/pvo5Sq6d5IQgghhHCmQM9XSr0N1FZK3Q18Abzr3lhCCCGEfyv3GrRSSgEfAG0wXcNaA3/WWq/wQDYhhBDCb5VboLXWWim1RGvdEZCiLIQQQniIM0Pc65VSMlVXCCGE8CBn1kH3BMYppfZjZnIrzMl1J7cmE0IIIfyYMwV6gNtTCCGEEOI8FQ5xa633A7WBoY5bbcd9QgghhHCTCgu0UuoBYDbQwHGbpZSqcEcEpVRTpdTXSqltSqmtjvcRQgghhBOcGeK+C+iptc4BUEq9CPwI/LuC1xUB/6e1Xq+UCgfWKaVWaK23XVZiIYQQwg84M4tbAcVlvi923FcurfUhrfV6x5/PANuBqKqEFEIIIfyNM2fQ7wM/KaU+cXw/AnivMgdRSsUAXYCfKvM6IYQQwl9VWKC11q8qpVYC1zjuukNrvcHZAyilwoAFwINa69MXefwe4B6AZs2aOfu2QgghhE9zZrvJK4GtpcPVSqlaSqmeWusKz4aVUsGY4jxba/3xxZ6jtX4HeAege/fuujLhhRBCCF/lzDXot4DsMt9nO+4rl6OP93vAdq31q1WLJ4QQQvgnpyaJObabBEBrXYJz1657AROA65VSyY7bjVXMKYQQQvgVZwrtHqXUZH45a74P2FPRi7TWq3BitrcQQgghfs2ZM+jfAVcD6UAapjf3Pe4MJYQQQvg7Z2ZxHwVu9UAWIYQQQjg40+rzJcfM7WCl1JdKqUyl1HhPhBNCCCH8lTND3P0d65eHAPuAeOBhd4YSQggh/J0zBbp0GHww8KHWOsuNeYQQQgiBc7O4FyuldgB5wL1KqUgg372xhBBCCP/mzH7Qj2FmcXfXWhcCucBwdwcTQggh/JkzZ9BorU+U+XMOkOO2REIIIYRw6hq0EEIIITxMCrQQQghhQ86sg75JKRVR5vvaSqkRbk0lhBBC+DlnzqCfKbu0Smt9CnjGbYmEEEII4VSBvthznJpcJoQQQoiqcaZAr1VKvaqUinPcXgXWuTuYEEII4c+cKdB/AAqADxy3s8Dv3RlKCCGE8HfO7GaVAzzmgSxCCCGEcLhkgVZKva61flAptQjQFz6utR7m1mRCCCGEHyvvDHqm4+srnggihBBCiF9cskBrrdcppQKBe7TW4zyYSQghhPB75U4S01oXA82VUiEeyiOEEEIInFvPvAf4Xim1kDKbZGitX3VbKiGEEMLPOVOgdztuAUC4475fTRoTQgghhOs4U6C3aa0/LHuHUmq0m/IIIYQQAucalTzu5H1CCCGEcJHy1kEPAm4EopRSU8o8VAsocncwIYQQwp+VN8SdAawFhnF+7+0zwB/dGUoIIYTwd+Wtg94IbFRKzXE8r5nWeqfHkgkhhBB+zJlr0AOBZGAZgFIqwbHkSgghhBBu4kyBfhboAZwC0FonA7FuSySEEEIIpwp0odY664L7ZB20EEII4UbOrIPeqpQaCwQqpVoCk4Ef3BtLCCGE8G/OnEH/AWgPnAXmAqeBB92YSQghhPB7FZ5Ba61zgScdNyGEEEJ4QHmNSsqdqa21Hub6OEIIIYSA8s+grwIOYoa1fwKURxIJIYQQotwC3Qi4AbgNGAt8DszVWm/1RDAhhBDCn11ykpjWulhrvUxrfTtwJbALWKmUut9j6YQQQgg/Ve4kMaVUKDAYcxYdA0wBPnF/LCGEEMK/lTdJbAbQAVgCPKe13uKxVEIIIYSfK+8MejyQAzwATFbq3BwxBWitdS03ZxNCCCH8Vnm7WTnTxEQIIYQQbiBFWAghhLAhKdBCCCGEDUmBFkIIIWxICrQQQghhQ24r0EqpqUqpo0opWZ4lhBBCVJI7z6CnAQPd+P5CCCGEz3JbgdZafwuccNf7CyGEEL5MrkELIYQQNmR5gVZK3aOUWquUWpuZmWl1HCGEEMIWLC/QWut3tNbdtdbdIyMjrY4jhBBC2ILlBVoIIYQQv+bOZVZzgR+B1kqpNKXUXe46lhBCCOFryt0P+nJorW9z13sLIYQQvk6GuIUQQggbkgIthBBC2JAUaCGEEMKGpEALIYQQNuS2SWJCXI7iEk1hcYnjpikqLqHA8efC4hIKikooKn1OkXmsqPSxMq8x3//yvELHa2qGBNIhKoKOURHUCwu1+q8rhNNO5BSwOzOb3Uez2XU0m92Z2ZzKK+TRgW24skU9q+MJF5ICLS7L97uO8W1qJoVF5xfUX/58/vcFxZrCohKKSsz9BUXm/qISfa7QFhaXUKLdlzk4UFFY/MsBompXp1N0BB2jTcHuGBVB7Roh7gsgRAWKSzQZp/LOFeDdmaXFOIcTOQXnnlctOIAW9cPIyitk4tSfmXJrFwZ2aGRhcuFKUqBFlWit+c/K3byyfCdBAYrQoECCAxXBgQGOW5k/BwUQHGC+rx4SQEigIijAcX+gIiQwgCDH80Mcrzn/e+V4jwCCgy5xjAu+Dwk0zw0K+OXPwYEBBAUolFKczi9kS3oWm9Oy2Jxubku3HD7392tWtwYdoyPoFGUKd4eoCGpVC7bwJy58UX5hMXsyc9jlOCMuLcR7j+Vwtqjk3PPq1QwhLjKMAe0bERdZk7gGYcRHhhFVuzoBAYqTOQXcMW0N981exz9u6sitPZpZ+LcSrqK0duOpSiV1795dr1271uoYogJ5BcU8/NFGFm86xPCEJrw4shPVggOtjnXZTuUWsCX9NJvST7ElPYtNaVmkncw793hs/Zp0jIowZ9tREbSPiiAsVD7jioodzz7L7sycMmfC5pZ2Mo/SX8EBCprWrUFcZBhxkTWJbxDm+HMYdWpWPKKTW1DEvbPW801KJg/1b8Xv+8ajlHLz30xcLqXUOq1194s+JgVaVEbGqTzumbmWrRmneXRgG37bp4VP/xI4kVNgzrDTTjm+ZpGRlQ+AUtCifk06Rdc+V7jbNalFjRAp2v6ouESTdjLXcX0457xCfDK38NzzSoelzxXgBqYYx9SredkfdAuLS3j4w418mpzBpKtj+POQdgQE+O7/n75ACrRwiXX7T/DbmevILyxhym0JXN+modWRLJF55uy5M2wzPH6KI6fPAuYsqGWDcDqUnmlHR9CucS2fGGEQRl5BMXuO/XJNuHTC1p5jORSUGZauHxZCC8cZsCnGNYkrMyztLiUlmr8v2c57q/YytHMT/jm6MyFBsmDHrqRAi8s2f81Bnvx0M1G1q/O/27sT3yDc6ki2cuR0PpvTstiUnuUo3qc4lm0m8wQGKFo1DKdjVC06RtemU1QEbRqHExokRduutNYczykwM6UdZ8Slw9Ppp3657FE6LB0fGUZcg1+GplvUd25Y2p35//vNHl5ctoPeLevz3/HdqCmXYy5LQVEJ/1yxk3E9mtOsXg2Xva8UaFFlRcUl/O3z7Uz7YR+9W9bnjdu6ElFDJktVRGvN4dP55iy7zES00hm4wYGK1o3C6RhV+9w17VYNw+VMx0Jaa5ZsPsy0H/aSejSbUxcMS8eddzZsvjavV8PWoyPz1xzksY830TEqgqmTrpAlhVWUcSqP389Zz4YDp3hmaDvu6BXrsveWAi2q5FRuAffP2cCqXce465pYHh/UhqBAKSBVpbUm/VTeBWfaWWTlmUIQEhhA28bhjtnjtekQFUHLhmEEy8/c7XYePsOzC7fy457jxEXWpGeLeucNTTeJcO+wtDut2HaE++esJ6p2dWbc1YPoOq47+/MH36Vm8sC8ZAqKSnhpVCdu7NjYpe8vBVpUWuqRM/xmxloOncrnbzd1YEz3plZH8klaaw6eyGNT+qlfzrTTsjhztgiA0KAA2jWpRaeoCHq2qMeA9o0I9NJCYUdZuYW89kUKM1fvJ7xaEP/XvzVjezTzuZ/xz3tPcNf0NdQICWTGnT1p3UguUVWkpETzxte7eO2LFFo2COOt8d2Iiwxz+XGkQItK+WLbER78IJlqwYG8PaEb3ZrXsTqSXykp0ew/kcumtF+K9pb0LHIKimnTKJxHBramb+sGPj173t2KSzQfrj3IS0k7OZVbwNiezfi/G1pbet3Y3XYcPs3E934mv7CYqZOuoHtMXasj2dbJnAL+OD+ZlTszualLFH+/qYPbVmdIgRZOKdt8pEOTCN6Z2I3GEdWtjiUwBWXZlsO8nLSDfcdz6RFTl0cHtZEPT1Wwbv9Jnl24lc3pWVwRU4dnhranQ1SE1bE84uCJXCZO/ZmMU3n8Z1xX+rX1z5UY5dl48BT3zV5P5pmz/HloO8b1bObWD8NSoEWF8gqKeWTBJhZtzGBY5ya8NMo3mo/4msLiEj5Yc5B/fZlK5pmz3NCuIY8MaE3LhjJkWZGjp/N5YdkOPl6fTsNaoTxxY1uGdW7idyMRx7PPcse0NWzNOM0LN3dktFy+AswJyqyfDvDXRduIDA/lP+O60rlpbbcfVwq0KFfZ5iMPD2jNvdfG+d0vLW+TW1DE1FV7efubPeQUFDGqWzQPJraiSW0Z8bhQQVEJ037Yy5Qvd1FQVMJdvWO5v2+8Xy87yj5bxO9mrmPVrmM8PqgNv702zupIlsotKOLJT7bwyYZ0rmsdyWtjEjx2uUMKtLgk03xkPfmFxfzr1gQZ8vIyJ3MKePPrXcz4cT8omHR1DPddFyebfTh8k5LJc4u2siczh+vbNODpIe2IrV/T6li2UFBUwp/mJ7N40yHu7h3L44Paeu1M9cuxOzObe2etI/VoNn9MbMX9feM9+nOQAi0uav6agzz16Raa1K7GuxO7yzCpF0s7mctrK1L5eEMaYaFB3HtdHHdcHUv1EP+8THHgeC5//XwbK7YdIaZeDf48tJ3fdr4rT0mJ5rlFW5n+435u7hLFi6M6+dWyviWbD/HIR5sIDlRMua0LvVtGejyDFGhxnqLiEv6+ZDvvf7+Pa+Lr88bYLnLG5SN2Hj7Dy0k7+GL7URrWCuWBfq0Y0z3ab9av5xYU8dbK3bz97R6CAhR/uL4ld14TI13byqG15o2vdvHPFSn0bR3Jm+O6+nw/+cLiEp5fsoOp3++lS7PavDm2q2WXh6RAi3PKNh+5o1cMT97Y1m9+efuTNftO8MLSHazbf5IW9Wvy8IDWDOzQyGfnFmit+XzzIf7x+XYysvIZkdCExwa1pVFENaujeY05Px3gqU8307lpbd6fdIXPfmg/nJXP/XPWs3b/SSZdHcMTN7a1tIOfFGgBwK6jZ/jN9LWkn8rj7yM6MuYKmb3py7TWfLH9KC8t20Hq0Ww6N63NowNbc3VcfaujudSOw6d5duFWVu85QbvGtXhueHuukDW+VbJsyyEmz0umed0azLirh88ts/xh1zEmz9tAbkExL4zsxLDOTayOJAVawJfbj/DAvNLmI13p1lx+gfmL4hLNx+vTeG1FChlZ+fRpFcmjA1vTvol3r/3Nyi3k1RU7mbl6P7WqB/NQ/9bc5oNdwDztx93HuWfGWsKrBTHjrh4+sTFOSYnmrW9288/lO2kRGcZ/x3e1zd9LCrQf09r8w3w5aSftm9TinQndZSmOn8ovLGbmj/t5c+UuTuUWMjyhCf93Q2uX7szjCcUlmg/WHOTlpB1k5RUyrmdz/q9/K58dkrXC1owsbp+6hqKSEt6fdAVdmnlvQ5ys3EL+ND+ZL3ccZVjnJjx/c0dbLbHzywK9aGMGcZFhtGtSyyXv543yC4t55KNNLNyYwdDOTXhpZCe/ndUrfpGVV8g73+7mvVV7KS7RjO3RjD/0a0l9L9jpaN3+EzyzcCtb0k/TI6Yuzw5r79f/j7vT/uM5TJz6M0dPn+Wt8V25rnUDqyNV2pb0LO6dvY7DWfk8NbgdE69qbrt5GH5XoIuKS+jxjy85kVNA28a1GNk1iuEJUUSG2/8XkKscysrjnhnr2JKRxUP9W3PfddJ8RJzvyOl8/vVlKh+sOUi1oAB+07sFd/dpQZiNzi5KHT2dzwtLd/DxhnQa1arGE4PbMrRTY/k37WaZZ85y+9SfSTlyhldGd2ZElyirIzlFa828NQd5ZuFW6tcM4Y1xXelq01EAvyvQYBo4LNqUwYJ1aWxMyyIwQHFdq0hGdoumX9sGPr3sYt3+k/x25jryC4t5/ZYEEtvJ+k9xabszs/nn8p0s2XyYejVDuP/6eMb2bGaL/0cKikp4//u9TPkylcJizd19YrnvOv/uAuZpp/MLuWfGWlbvOcHTQ9px1zWu2wvZHfIKinnq0y0sWJ9G75b1+detXahr401Q/LJAl5V65AwfrU/j0w3pHDl9lojqwQzt3JhR3ZrSOTrCpz6Fz197kKc+2ULj2tX4nzQfEZWw8eApXli6gx/3HKdp3er83w2tGda5iWXdpVbuPMpfFm1jz7EcEts24KnB7YiRLmCWyC8s5sF5ySzbeph7r4vjkQGtbfl7c9+xHH43ax07j5zhD9e35IF+LW0/adDvC3Sp4hLNql3HWLAujaSthzlbVEJcZE1Gdovmpi5RXr2koKi4hOeX7uC9VXvpFV+PN8d2lUkzotK01nybeowXl+5g26HTtG1ci0cGtua6VpEe+4W8/3gOf128nS+2HyG2fk3+PLQdfb3w+qevKS7RPP3ZFub8dIAx3aP5x00dbdVDYdmWwzz84UYCAxWv35LgNdfMpUBfxOn8QpZsOsSC9Wms2XcSpeCa+PqM7BrNgPaNvGoyVVZuIffPXc93qceYdHUMTw2W5iPi8pSUaBZtyuCfy1M4cCKXK1vU5dGBbdw6mze3oIg3v97Fu9/uJThQ8Yd+LbmzV6ylTSTE+bTWvPZFKlO+TCWxbUPeGNvF8l3viopLeClpJ+98u4fO0RG8Oa4r0XW8Z2WCFOgK7DuWw8fr01iwPp30U3mEhQZxY8dGjOwaTY/YurYcyiklzUeEOxUUlTD35wP8+6tUjmUXMLB9Ix4a0Jr4BmEuO4bWmkWbDvH8ku0cysrnpi5RPDaoDQ1rSRcwu5rx4z6eWbiVK5rX5d3buxNRPdiSHEdP53P/3A38vPcEE65szlND2tpi7kRlSIF2UkmJ5qe9J1iwPo0lmw+RW1BMs7o1uLlrFCO7RtO0rr0+lX214wiT5yZTLTiA/47vRnfpniTcJOdsEf/7bi/vfLub/KISxnSP5oF+rS67leb2Q6d5ZuFWft57gvZNavHcsPby79hLLN6UwR8/SCYuMozpd/bw+Aeq1XuOc/+cDeScLeL5mzt6zQzzC0mBroKcs0Us23KYBevT+HHPcbSGHrF1GdU1mhs7NbZ0KYrWmv9+s4eXknbQrnEt3pnYnShpPiI84Fj2Wd74ahezf9pPgFLc0SuWe6+NI6JG5c6gTuUW8OqKFGat3k9E9WAeHtCGW65oavsJPeJ8q1KP8duZa6lTM4QZd/agRaTrRlYuRWvN29/u4eWknTSvV4O3xnWjdSPvnQwrBfoypZ/K4xPHEPjeYzlUCw5gYPtGjOwWzdVx9T36SyW/sJhHF2zis+QMhnRqzMujOnvV9XLhGw6eyOXVFSl8mpxOrWrB3HddHLdfHVPh9cjiEs28NQd4JWknWXmFjL+yOX+6QbqAebNNaae44/01AEy7owcdo93XQjYrr5CHPtzIim1HGNyxMS+O6mTLdfuVIQXaRbTWrD9wigXr01i0MYMz+UU0jqjGTV2iGNktmjg3f3o8lJXHb2euY3O6NB8R9rAt4zQvJe1g5c5MGkdU48HElozsevHtLdfuM13Atmacpmes6QLWtrF0AfMFezKzmfDez5zKLeDtCd25pqXrN2TZmpHFfbPXk34yj8dvbMudvWJ84vefFGg3yC8sZsW2IyxYn8a3KZmUaEhoWpuR3aIZ1qlJpYf8KrL+gGk+knu2iNdv7cIN0nxE2MjqPcd5YekOkg+eIr5BGA8PaE3/dg1RSnHE0QXskw3pNI6oxhM3tmWIdAHzOUdO53P71J/ZnZnNa7ckMKST63aKmr/2IE9/uoXaNYJ5c2xXn5qnIAXazY6ezufT5HQWrEtn55EzhAQGcEO7hozsFkWflpGXveTpo3VpPPHxZhpFVON/t3enlTQfETaktSZp6xFeStrBnswcujarTa/4+kxdtZfCYs09fVpwX984aoR495CkuLSsvEJ+M30Na/ef5Llh7Zl4VcxlvV9+YTHPfLaVD9Ye5Oq4eky5rYtX9IyvDCnQHqK1ZmvGaT5al8ZnyemczC2kflgoIxKaMLJbdKWH84qKS3hh6Q7+t2ovV8eZ5iN1bNyyTggw/24/WpfG61+kcvh0PoltG/L0kLY0ryddwPxBfmEx98/ZwBfbjzC5X0v+mNiySqMlB47ncu/sdWzNOM39feP54w2tfHISoRRoCxQUlfD1zqMsWJfGVzuOUlSiade4FiO7RTM8oUmFnwIvbD7y5OC2BEvzEeFF8guLOZyVL+05/VBRcQlPfLKZ+WvTGNuzGX8d3qFSxXXFtiP8aX4yCnjtlgT6tfXdS3pSoC12IqeAhcnpLFifzub0LIICFNe1bsCoblH0bfPrjTt2HT3D3TPWkXYyl78O78CtPZpZlFwIIapGa81LSTt5a+VuBnVoxGu3JFQ4y7+ouIR/rkjhrZW76RBVi7fGdbNd/wlXkwJtIzsPn2HB+jQ+2ZBO5pmz1K4RzLDOTRjZNZpO0RGs3JnJ5LkbCA0O4K3x3bjChyZDCCH8z3ur9vLXxdu4skVd3p3YnfBqF59Am3nmLJPnbuDHPce5rUcznhnazvI2op4gBdqGiopL+M6xccfybUcoKCohpl4N9p/IleYjQgif8umGdB76cCOtG4Uz7Y4eRIaff4lvzb4T/H72erLyCvn7TR0Z1S3aoqSeJwXa5rLyCvl80yEWbkw3u/cMaS/NR4QQPmXlzqPcO2s9DWqFMvPOnjSrVwOtNe+t2svzS3fQtE513hrfze/WxkuBFkIIYbkNB05yx7Q1BAUE8ObYLkz7YR9LtxxmQPuGvDy6M7UuMfzty6RACyGEsIVdR88w8b2fycjKJzBA8djANvymd6zfNq4pr0C7dd2OUmqgUmqnUmqXUuoxdx5LCCGE/cU3COeje6/m5q5RzPlNT+7u08Jvi3NF3NbSRykVCLwJ3ACkAWuUUgu11tvcdUwhhBD216R2dV4dk2B1DNtz5xl0D2CX1nqP1roAmAcMd+PxhBBCCJ/hzgIdBRws832a477zKKXuUUqtVUqtzczMdGMcIYQQwntY3jtSa/2O1rq71rp7ZGSk1XGEEEIIW3BngU4Hmpb5PtpxnxBCCCEq4M4CvQZoqZSKVUqFALcCC914PCGEEMJnuG0Wt9a6SCl1P5AEBAJTtdZb3XU8IYQQwpe4ded0rfUSYIk7jyGEEEL4IssniQkhhBDi16RACyGEEDYkBVoIIYSwISnQQgghhA1JgRZCCCFsSAq0EEIIYUO22g9aKZUJ7HfhW9YHjrnw/cTFyc/ZM+Tn7Bnyc/Yc+VlDc631Rftc26pAu5pSau2lNsIWriM/Z8+Qn7NnyM/Zc+RnXT4Z4hZCCCFsSAq0EEIIYUO+XqDfsTqAn5Cfs2fIz9kz5OfsOfKzLodPX4MWQgghvJWvn0ELIYQQXsknC7RSaqBSaqdSapdS6jGr8/gqpVRTpdTXSqltSqmtSqkHrM7ky5RSgUqpDUqpxVZn8VVKqdpKqY+UUjuUUtuVUldZnckXKaX+6PidsUUpNVcpVc3qTHbkcwVaKRUIvAkMAtoBtyml2lmbymcVAf+ntW4HXAn8Xn7WbvUAsN3qED7uX8AyrXUboDPy83Y5pVQUMBnorrXuAAQCt1qbyp58rkADPYBdWus9WusCYB4w3OJMPklrfUhrvd7x5zOYX2ZR1qbyTUqpaGAw8D+rs/gqpVQE0Ad4D0BrXaC1PmVpKN8VBFRXSgUBNYAMi/PYki8W6CjgYJnv05Ci4XZKqRigC/CTxVF81evAI0CJxTl8WSyQCbzvuJTwP6VUTatD+RqtdTrwCnAAOARkaa2XW5vKnnyxQAsPU0qFAQuAB7XWp63O42uUUkOAo1rrdVZn8XFBQFfgLa11FyAHkDksLqaUqoMZ1YwFmgA1lVLjrU1lT75YoNOBpmW+j3bcJ9xAKRWMKc6ztdYfW53HR/UChiml9mEu2VyvlJplbSSflAakaa1LR4E+whRs4VqJwF6tdabWuhD4GLja4ky25IsFeg3QUikVq5QKwUw+WGhxJp+klFKY63XbtdavWp3HV2mtH9daR2utYzD/nr/SWssZh4tprQ8DB5VSrR139QO2WRjJVx0ArlRK1XD8DumHTMa7qCCrA7ia1rpIKXU/kISZHThVa73V4li+qhcwAdislEp23PeE1nqJdZGEuCx/AGY7PtzvAe6wOI/P0Vr/pJT6CFiPWQmyAekodlHSSUwIIYSwIV8c4hZCCCG8nhRoIYQQwoakQAshhBA2JAVaCCGEsCEp0EIIIYQNSYEWQgghbEgKtBAWUUoVK6WSy9xiynnuNKXUqIvcf52z208qpWKUUnll1qyjlJqqlDqqlNpSzutGVHaXMqVUH6XUeqVU0YW5lVK3K6VSHbfby9z/tVIqWynVvTLHEsJXSYEWwjp5WuuEMrd9Hjjmbq11QpnvpwEDK3jNCMzWrZVxAJgEzCl7p1KqLvAM0BOz89wzjt7MaK37AmsreRwhfJYUaCFsRCmVoJRarZTapJT6pLR4XfCcgUqpHUqp9cDNZe6/tszZ+AalVHhFx9NafwucKCfP1cAw4GXH+8Y5k1FrvU9rvYlf7741AFihtT6htT4JrKDiDwhC+CUp0EJYp3qZgvqJ474ZwKNa607AZszZ5jlKqWrAu8BQoBvQqMzDDwG/d5wh9wbyLjeg1voHTC/7hx1n+bsrylgB2Q5WCCdJgRbCOmWHuG9SSkUAtbXW3zgenw70ueA1bTA7AaVq06e37K5W3wOvKqUmO96nyNWBncwohHABKdBC+Ait9QvAb4DqwPdKqTYWR7oY2Q5WCCdJgRbCJrTWWcBJpVRvx10TgG8ueNoOIEYpFef4/rbSB5RScVrrzVrrFzHbrrqqQJ8BwiuRsTxJQH+lVB3Htev+jvuEEBeQAi2EvdyOmZC1CUgA/lL2Qa11PnAP8LljktjRMg8/qJTa4nhtIbC0ooMppeYCPwKtlVJpSqm7LvK0ecDDjolncRVldLzvFUqpNGA08LZSaqsj/wngr5gPEGuAvzjuE0JcQLabFMJPONZZL9Zad7A6y6UopVYCD2mtZbmV8HtyBi2E/ygGIso2KrETpdTXQAvM2b8Qfk/OoIUQQggbkjNoIYQQwoakQAshhBA2JAVaCCGEsCEp0EIIIYQNSYEWQgghbOj/AVKxgRmgkV69AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(8, 6))\n", "plt.plot(acc, label = 'R-squared')\n", "plt.legend()\n", "plt.plot(-mae, label = 'MAE')\n", "plt.legend(prop={'size': 12}, loc=9)\n", "plt.xlabel('Folds [1 to 10]')\n", "plt.ylabel('Metric score [i.e., R-squared 0 to 1]')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "We can also look at the **overall accuracy** of the model" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "R2: 0.5544107024745264\n", "MAE: 3.7082530497804926\n" ] } ], "source": [ "from sklearn.metrics import r2_score\n", "\n", "overall_acc = r2_score(y_train, y_pred)\n", "overall_mae = mean_absolute_error(y_train, y_pred)\n", "\n", "print('R2:', overall_acc)\n", "print('MAE:', overall_mae)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now, let's look at the final overall model prediction" ] }, { "cell_type": "code", "execution_count": 86, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Predicted Age')" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4v0lEQVR4nO3de3jcdZnw//c9x2SSNIemSUPbNOFQsB5ArLIIW7rBdTcuz4I8Xj66B1gE0R9bV9HNLrrXhZqqF1qFda2ri5R9cFdFXfFZBOIJKZWFxS0KKAUKNCltSZO0TdNkJpnj/ftjDkzSHGaSOWbu13X1ajIz+X4/00k/9/f7Ody3qCrGGGMql6PYDTDGGFNcFgiMMabCWSAwxpgKZ4HAGGMqnAUCY4ypcK5iNyAbzc3N2tHRUexmGGNMWXniiSeOquqquZ4vq0DQ0dHBnj17it0MY4wpKyJyYL7nbWjIGGMqnAUCY4ypcBYIjDGmwlkgMMaYCmeBwBhjKpwFAmNMWenr66Orq4vOzk66urro6+srdpPKngUCY0zZ6OvrY+vWrQwODtLU1MTg4CBbt261YLBEFgiMMWVj+/bteDweampqEBFqamrweDxs37692E0ra3kPBCJSJSK/EpGnROQZEfl04vFOEXlcRF4Uke+KiCffbTHGlLf+/n58Pt+0x3w+HwMDA8Vp0DJRiDuCINClqucC5wF/LCK/B3weuE1VzwRGgWsL0BZjTBnr7OwkEAhMeywQCLDcU8/ku4BY3gOBxk0kvnUn/ijQBfxH4vG7gCvy3RZjTHnr6ekhFArh9/tRVfx+P6FQiJ6enmI3LS8CoQhHxqYYDYTzep6CzBGIiFNEngSGgZ8BLwEnVDWSeMkhYM0cP3u9iOwRkT0jIyOFaK4xpkR1d3ezY8cO2traGB0dpa2tjR07dtDd3V3spuVMNKaMBcIcPB7gyNgUgVBk4R9aooIknVPVKHCeiDQAPwTOyeJnbwduB9i0aZMVWDamwnV3dy+rjj9pKhzl5FQYfzCa96GgmQqafVRVT4jIQ8CFQIOIuBJ3BWuBw4VsizHGFJuqMhGMcHIqQjAcLVo7CrFqaFXiTgARqQb+EHgWeAh4V+JlVwP/me+2GGNMKQhHYxybCPLy8QAj48GiBgEozB1BG3CXiDiJB57vqep9IrIXuFtEPgP8BthZgLYYY0zR+IMRxqciBRn3z0beA4GqPg28cZbH9wNvyff5jTGmmKIxZXwqzPhUhHA0VuzmzKqsKpQZY0y5KObkb7YsEBhjTI6oKuPBCCcnw4QipXn1PxsLBMYYs0ShSIyTU2EmpiLESvzqfzYWCIwxZpH8wQgnp8JMhoq76mepLBAYY0wWItEY41Px1T+RWPkM/8zHAoExxmRgKhzl5GQYf6j0J3+zZYHAGGPmEIu9Ovlbqks/c8ECgTHGzFBOSz9zwQKBMcYQv/qfCJXf0s9csEBgjKlo5b70MxesZrExJq/6+vro6uqis7OTrq6ukik0PxGM8MqJSQ6NBjg5Ga7YIAAWCIwxedTX18fWrVsZHBykqamJwcFBtm7dWrRgEIspJwIhXj4WYPjkFFNFzvpZKiwQGGPyZvv27Xg8HmpqahARampq8Hg8bN++vaDtCEaijIwHOXA8wHF/aNms/88VCwTGmLzp7+/H5/NNe8zn8zEwMJD3cz/wwANsedsf0bHxjVz6p+/m/p8+WBErgBbDAoExJm86OzsJBALTHgsEAnR0dOTtnJFojO/f+wBbb/o0gycCNKxcxcjwML29vezevTtv582HyXCUXc8P83f/8RT/+WT+ijhaIDBmAaU62VkOenp6CIVC+P1+VBW/308oFKKnpyfn55oMRRk6OcXLxwN8+es7cXs8+Hw+RASfz4fb7eaOnaVf/yrZ+X/qR89w5T8/Su99z/LzZ4f54W/KOBCIyDoReUhE9orIMyLy4cTjnxKRwyLyZOLPO/LdFmOyVWqTneWmu7ubHTt20NbWxujoKG1tbezYsSNnxedjMWVsMszB4wEGxybxB+OVvw4dPER1dfW011ZXV3P40KGcnDfXZuv8d+87SjASw+UQLj6zmSvOW5O380u+x8xEpA1oU9Vfi0gd8ARwBfBuYEJVv5jpsTZt2qR79uzJT0ONmUVXVxeDg4PU1NSkHvP7/bS1tfGLX/yiiC2rbAut/b/q6qsZGR6eNj8RCARY1dLCN++6q5BNndNkOMrj+4+xa98Ij+8/TjBtE5vLIbxpfSOXbFjFRWeuZF1TDU01nkWfS0SeUNVNcz1fiFKVg8Bg4utxEXkWyF9oMyaH+vv7aWpqmvZYoSY7zakCoQgnJxeu+XvdtdfS29tLIBCgurqayclJwuEw1117bYFaOruFOv/z1zeyJdH511W5C9augu4sFpEO4vWLHwcuAraKyFXAHuBjqjo6y89cD1wP0N7eXrjGGkN8snPmHUG+JzvNdLGYMj4Vz/ufaeK3zZs3c/PNN3PHzp0cPnSINWvXct2117J58+Y8t/ZUC3X+mzriV/5vPaOwnX+6vA8NpU4kUgs8DHxWVe8RkVbgKKDANuLDR++b7xg2NGQKLTlH4ElMPAYCAUKhUE7Huc3sQpEYY5Nh/MHyS/0Q7/yPs2vf8JzDPlvOXsVFZzRTW7Xw9XiDz1PeQ0OJRriBHwDfUtV7AFR1KO35bwD3FaItxmQjOdm5fft2BgYG6OjooKenp6SCQF9fH9u3b6e/v5/Ozs6CtS9f5y3Xql/ZjPkX68p/LoWYLBbgLuC4qn4k7fG2xPwBInIjcIGqvme+Y9kdgTHTFeuOJdfnjcaUiSyHf0pB8sr/4X0j/Pf+Y3nr/PN9R1CIQHAx8Evgt0DyX+kTwHuB84gPDQ0AH0gGhrlYIDBmumKtasrVeYORKCcnI0wEI2Wz6zeTYZ9cX/mX/dCQqj4CyCxPPZDvcxuz3BVrVdNSzquq+EPxso/lkvRtoc6/WKt9csXqERhTxoq1qmkx5y23ou/FuPIvFgsExpSxnp4etm7dCjBtrH62FA65nNzN5ryBULzzT+76LWXpY/6P7z/GVFrn70zr/C9eBp1/uoItH80FmyMw5lTJDn6+VU35mFSe77yLWftfLJl0/sUe9in7yeJcskBgzOIUalK5XNb+l0Pnn67sJ4uNMcWX70nlclj7X26dfyFZIDCmAuRjUrkc1v5b558ZCwTGlKje3l5uu+02xsfHqaur48Ybb+Tmm29e1LGymdxdSKmv/bfOP3sWCIwpQb29vWzbtg2Hw4HH4yEQCLBt2zaARQWDXKTKKOXhH+v8l8Ymi40pQY2NjQQCAVyuV6/VIpEIPp+P0dFTkvTmTTSmjE+FGZ+KlNzwz1Q4yuP9x9n1/PLv/G2y2JgKND4+jscz/T++0+lkYmKiIOfPZPhn9+7d3LFzJ4cOHmLtusKkea6kzh/A5XBQ5XZQ7Xbm9zx5PboxZlHq6upOuSOIRqPU1tbm9byZDv/s3r2b3t5e3G43DQ31qeLwN998c86DwWQ4yq8W6Pwv2bCKi85YyYrq8u783U4HVW4nVe74325nYcrKWyAwpgTdeOONbNu2jUgkgtPpJBqNEovFuPHGG0957VJ3DCeHf05OZp764Y6dO3G73alSkMkJ6Dt27sxJIMh0h2+5d/7Jjr/a46TK5cBVoI5/JgsExpSg5ITwbbfdxsTEBLW1tbOuGkrfMdzU1MTg4CBbt27NaMfwVDjKyakw/mA069U/hw4eoqGhftpjSy0OXwkTvi6HgypPovMv4BX/Qmyy2Jgylu2O4VhMmUjk/gkuIfNnrorDL/crf6dDEkM98Y7f4ypOx2+TxcYsY5nuGA5GooxPRZiYyk3qh6UUh89kwrdcO3+HSKrTr/I48LryO8mbKxYIjClj8+0YVlUmgvGr/6lwNKerfLItDp/s/B9+Pl7Jazl1/l63E19inN/rchAvylheKnpoqFi1Xo3Jlb6+Pq655hrGx8eJRCK4XC7qVtTzlX+5g7dcdAnRWPz/d/oqn/Qr+Hys8klablf+qUD68gHaT1vNjR+6gT/9k26cjtLv+Is+NCQi64BvAq3Ey1LerqpfFpEm4LtAB/FSle9W1YLtlFnKJJsxpSR5BapONw5fPY66ZvxhUkEA8r/KJ2m5df4Q//f970cepvcf/g6PA1ZWexk5+BJ/+5EPUeVaHv1FIWoWtwFtqvprEakDngCuAP6KeEH7W0TkJqBRVf9+vmPl8o6gWLVejcmlP+jqYnDkONUrVoIzfl0326RtV9elNDTUTxu2UFXGxsZ48MEHl9SGjIZ9zmrmojOby6bz97gc+Dyu+Fi/28Gll15a1v1F0e8IEgXpBxNfj4vIs8Aa4HJgS+JldwG7gHkDQS4Vq9ZrObAhs9IXicY4ORWhf3ichsYWSOvgZ1vGuXbd2lNW+UxOTrJm7dpFnX85jfnv3r2bnXd8g4MDL9Gx5jR6Pvo3/Mk73jHtNcu9vyjoZLGIdABvBB4HWhNBAuAI8aGj2X7meuB6gPb29py1pVi1XkudDZmVtqlwvOi7PxRf+7+2vT2jDn4pq3zSz71cOn8Rocrt4L8efojem27E63Kw0udj6PAB/uZDH8IhMu33vZj9RSEuzAo2WSwitcDDwGdV9R4ROaGqDWnPj6pq43zHyOXQUD5K9y0HNmRWmiaCEU5OhpmasfY/m0ng5GRnJqt8kpZT5+92Oqj2OPF5nFS5nDgckvHve7H6i1ydtyRKVYqIG7gP+Imq3pp47Hlgi6oOJuYRdqnq2fMdJ1+rhhablnc56uzspKmp6ZSx5NHRUfbv31/EllWeTFM/LKaDn8+CnX97A5ec3ZLzzj/XSeySm7mqPXPv4s3m970Y/UWuLsyKHggk/i98F/GJ4Y+kPb4dOJY2Wdykqn8337FsZ3H+2R1B8YUiMU5OhXO2+SsTGXX+G1blbcI3V8tbk5u5qj3x3bwLKfXf91xdmBV9shi4CPhL4Lci8mTisU8AtwDfE5FrgQPAuwvQFrOAXFayMtkJhCKcnIwQCEXycvyZV9xX/dX7cK/ZWLTOP91il7cmk7b5Eh1/tmv6S/33vVBzExW9oczMzobMCicWU8YT4//5LPySvOJ2eatxnfYaphpOJ9Z8BrherXmQz2GfhWS6vNXpkET6htwlbSvl3/dlNUeQKxYIzHIRisRSlb/yPfwzFY7yFx/9NCeqTiO26kxwpnXwGuMtnSsLduU/l7mS2LWsWsX37v5WYj1/ZsM9y00uApUFAmMKZKFlfqpKIBTNuO7vUiZPpxLFXB7eN8Jj+48xFU6724hFcY0dxH30BSYHfsMvfnx/1u8116bNEXjdTE2MEZr0s+Mfb+UdM9b0m+xZIDCmAOa7hf/Dt/8R41Px5G+ZFn5ZzOTpfBO+aAw51k/1iX7cx1/CEQkuKm10PiTH+R95+EH+6dYvcWCgv+SGaMqdBQJjCmC21ScTUyFa1qznrm/dnXXhl0zz/We62kdG9vHFz20raNK5ueRjnN/MrxRWDRmz7CVTECiCuryoq4qqagcHB4ezDgIwfwWwqbQavpmv9mnD55Kc7jfIVKkUZzFzs0BgTA50nH4Gg8fGqK5rTOX9mQwEFp3LZ2ZuIHW4mPC1UfXac7nynx89pfM/v70hUcZx7gnfzZs3F6zjr3Y78VrHXzYsEBizBP5ghJNTYa6+4aP09vaik5OLzuWT7rprr+XTn/kc0dp1SNtGwo2d4PIQAYjEMu78C8E6/vK34ByBiGwAvkY8SdzrROQNwJ+q6mcK0cB0NkdgSsFcqR9ykeohfbXPIy+MkL7YxyGkFXAvXufvcjiocjuoSuTssY6/9C15slhEHgZ6gH9R1TcmHvudqr4upy3NwGICgaVUNrkSjEQ5ORlhIhhZ1Lj/XOZb6lkKV/4uh4MqT3xlj3X85SkXk8U+Vf3VjDqc+dkDn2OWUtkslariD0Vnzfy5FKXc+ad3/LaqpzJkEgiOisgZxMtMIiLvIlFoptRt374dj8eTWtKX/Hv79u0WCMy8Ms38mY1S7fwzydJplrdMAsFfA7cD54jIYaAf+Iu8tipHlntVofkUakhsuQ29TYXjO3/9wWhOhn9KsfO3yV0zU8YbykSkBnCo6nh+mzS3bOcISj3FbL4UqojGcinuo6rxwi9TEYI5GP4ptc7fIcl1/I6KzddT6XIxWfzRWR4eA55Q1SeX1rzsZBsIlktHla1CBcByD7TJxG8TwQjR2NKu/oPhKI8PxHf4FrvzT23gcjnxuh14XQ5mzPGZCpOLyeJNiT8/Snx/GfA08EER+b6qfmHpzcyP7u5uduzYUbIpZvOlUENi5Tj0Fosp/lA8789SJ39L5crfIYLPEx/qqXI78LrK84p/uQ0zlpNMAsFa4HxVnQAQkU8C9wObgSeAkg0EEA8GlfbLVKhiFsUs6J2tqXCU8akI/uDS0j5n0vkn0zvU57Hz97gcVLud+Dwuqtzlf8VvK/yKK5NA0AIE074PE99cNikiwTl+xhRRoaoulXp1p2hMmZiK7/xdStGXUhj28bgc0/L1ZFuJq9TZCr/iyiQQfAt4XET+M/H9/wK+nZg83rvQD4vIncSHk4aTm9BE5FPA+4GRxMs+oaoPZNl2M4dCDYmV6tBbLjZ+FbPzF5F4x5/W+S+3jn+mchxmXE4yWjUkIpuI1x4G+C9VzXjGVkQ2AxPAN2cEgglV/WI2jbUUE2Y+ybw/mRR9mU0mnX8+hn0cInjdDqpcztTqnnIf6slWOSw8KOc5jJykoU50/HsSdwFXisinVfVPMvzZ3SLSkVFrTcGU8y91uuTGr/GpyKKGf+br/JO5fTLt/DOtKJZczhlfy2+reqD0hxmX+xxGJstHPcCfAH8G/BHwA+AeVf3RvD84/RgdwH0z7gj+CjgJ7AE+pqqjc/zs9cD1AO3t7W86cOBApqc1c1gOy2onQ1HGp8L4Q9lv/Mpl5580X0WxSy65hCq3o6Lr7mailIvIl8Mdy3wWvY9ARN4OvBd4O/AQ8F3gK6rasYhGdDA9ELQCR4mnrdgGtKnq+xY6zlKHhpbLVfBSlesvdSgSYyIYYSKLko9JC3X+57c3suXsxQ/7nFJRLBphcvwEq1tW8tDPfjLtit9+D8tPZ2cnTU1N0z5HVWV0dJT9+/cXsWWZWcrQ0I+BXwIXq2p/4mBfzkWjVHUorYHfAO7LxXHns9xv7Waar7Mpp4m55K7fxaz7z3fnn+7Qyy/TWOdDguNINIyg1Ljh5f0vnhIEKun3cLkop6XSizFfIDgfeA/wcxHZD9wN5OSeVkTaVDWZuO6dwO9ycdz5VNLytIU6m3L4pV7suv98DPvMRkTwJtbyV3ucnN5Sl9G/aSX9Hi4npT6HsVSZrhp6K/Fhov8NPAX8UFVvz+gEIt8BtgDNwBDwycT35xEfGhoAPpAWGOa0lKGh5K3dyZMnGRoaIhgMpv5DHjlyZFHHLLRMhxQWGvop1TmCxa77L1Tn73U7qXI5qE4UZHE4Zr/Sn+/ftNyHGCpZKc9hLGTJuYZmHMwBvA14TyZj+rm2lEDQ1dXFCy+8wPDwMCKCw+EgGo3icDi45557Sv4DzabzzqSzWcovda7HuBcz8RsMR/nVwCi7nh/O27CP0yFUe+K7d33u6R3/bDL5Ny3X+RlT3nIaCIptKYGgr6+PK6+8klgshsvlIhaLEYvFaGlp4ayzzlrSf8JCTP5l0oEk2/HII4+kglx1dTWtra24XK6cdDa5uptYzLLP9Cv//95/nMm0OYNcdf7JZZ3Vnvys7inVuzGzvFkgSLN69Wr8fj+hUAiv10traysrVqxY0m15of5jL3SVn2xHKBRicHCQaDTeSbpc8Wmg5uZm7rzzziW3aalXtFPheLWvTK/+M5nwvWTDKi4+s5l6X/adf7IaV6ZX/blQzkMMpjzlZEPZcrFx48ZZO7GlTJIWavKvs7OTF154gbGxMYLBIF6vl/r6es4666xp7RgZGcHlcuF0OolEIkSjUaqrq2lpaQHiHflS7lwWs+Io27H/fK/28bqd1HjiV/3FyNRZiYkQTWmbMxCISNNczwGo6vHcNye/8jHzX6ilmFu2bOGXv/wlDocDp9NJMBjkyJEjvP/975/WjmAwiNPpRERwuVxEIhHOOeccDh8+nJNli9msOMpm7D+fwz7JilxVHic+txOXlWI0Zpr57gieIL6qR4B2YDTxdQPwMtCZ78blWj6SpBVqKeauXbtYvXr1KXcEu3bt4uabb061w+v1EgqFcDqdxGIxvF4vgUCAYDBIY2Pjku9cFgqm2Vz952vYR0SocjvwuV1Ueco3P78xhZJJiolvEF8u+kDi+27gClX9QAHaN00pJp0rxTmC4eHh1POtra14PB4mJiZYu3ZtTpYtzjbGveXStzMeXLjWb/pqn7mu/BfT+SdX+NR4XFQXaKzfmHKRi1KVv1XV1y/0WCGUYiCAwkz+ZbNqaO/evam7ho0bN9LT08P27dvn/Pnk89nOHWSa8iFfnb/XHR/qydcKH2OWi1wEgp8QTzXx74mH/hzYrKp/lLNWZqhUA0EhLPbOIz04jI2N0dzcTEtLS+rnr776au66666Mj5tpqcdk5//wvhEee+lYTjr/9HX9y7E4izH5kotA0ER8N/Bm4nMGu4HeYkwWV0IgmG9PQrZ3HjODx/DwMEePHqW+vj6jO4X05aCToSjjwTCBYHTOlA/5uPK3q35jli5n+whEpEZV/Tlr2SIs50DQ19fHTTfdxN69e/F4PLS1teF2u5c035DJcNJ8cw/P7XuRiWA8389cE7+57vztqt+Y3FvyPoJEnqE7gFqgXUTOJZ4b6IbcNbOyJa/cjxw5klrtc/DgQdrb2/F4PIvekzDb0tZwOMxjjz1GZ2cnnZ2drFixgkAgkAoWKg78oQhrN7yeQ6OBWY+b62Eft9NBjdeFz676jSmKTDaU3Ua8IM29AKr6VKL8ZEXq7e3ltttuY3x8PLU7WVWXlFoiuRksEomk9gAADA0NcdZZZy16T8LMTWgOh4NwOIzX603tJThx4gQqDtRdTVVdA5PBMGFx8L73T18UlsvOP315p8/rxG3r+o0pqkxLVR6cUUpvcUVhy1xvby/btm3D4XDgcDgIBAL09/fT1taW9Qat9LmAI0eOsGbNmml7AFSViYkJnn76aWpra+nr68s6yMzchDY1NYWq0tDQAA4X1Q3NxKobcXuraGhs5PChQ6xZ+2qJxVx2/m6nIzHkE8/lU+mlGY0pJZlMFv8HcCuwA7gA+DCwSVXfk//mTVfsOYLGxkYCgQAulyvVqQI4HA7OO++8jHPuzJzEfe6556ZdsYsIqoqI4Ha7aWlpwePxZD1XkMy4mrwjCEdjuKvrqF7RQOcZ8dQUqsrY2BgPPvggsPCV/xsTnf/vL9D5i0gqeZvPY1f9xhRTLnINfRD4MrAGOAz8FKjI+YHx8XE8Hg/AtE1TyQRvmaaWmJmfqKGhgVdeeYVYLIbH4yEUCgHxhHHt7e3U19fj9/uznivo7+9nVWsbq9Z0oE4P/S8fJBwOE4q8OvE7OTnJaevW88sXji75yj99U5fPY1f9xpSLTALB2ar65+kPiMhFwH/lp0mlq66uLnVHkLxqB3A64xOcmaaWmDmJOz4+jsvlSgWUZJ6gqqoq6uvrgezyF4WjMSamIqw9+1yGjx7D54nX0W1ububw4Vdwu12MTQQY86zC1XkhwfXn8sl7n0n9fDadf/pEr9flsM7fmDKUSSD4CvGylQs9NisRuRO4DBhOK17fBHwX6CBeoezdqjqaWZOL58Ybb2Tbtm2pSd1wOAxAS0tLKr11JgnsZuYnSiaKS9YO6O/vJxwOE4lEGBsbo76+fsEgE43Fa/tOBCMEE1f0177/enp7ewkEAlRXV+Nwe2naeCHu9nOJrDqLOnfVqweIxTi9wcEVF2yYNuyze/du7ti5k0MHD7F2XXz+4O2X/kH8qt8meo1ZFuacIxCRC4G3Ah8hvnIoaQXwTlU9N6MTxFcYTQDfTAsEXwCOq+otInIT0Kiqf7/QsYo9RwCvrhqamJjA4/HQ2toKkFVqidnmCMLhMM3NzRw7dgxVTc0VzDdHEE3s8vUHI0yGZp+/f3DXw3z9Bz/nmKcVaT0HdaZd3WsM19ghPEdfIHL4d7Q01vHNu+5KPb179256e3txu91Ue5xMjY8RCozz1R1fsTTKxpSRRW8oE5FLiNcW/iDw9bSnxoEfqeoLWTSiA7gvLRA8D2xR1UERaQN2qerZCx0n34GgEJXGZp5rYGCAuro6hoaGGB8fTw0PRaPRVBrp2tpavv3tb9Pd3Z1R5x8MR/mfgVF2zTHmHxt+ieqxfjzH9+OITAGnTho7RLjqz/8Pw68cosbrRoj/nlhZRWPKTy5STKxX1QNLbEQH0wPBCVVtSHwtwGjy+1l+9nrgeoD29vY3HTiwpKbMqdglBNNLaSb3J9TX16OqHD9xgqeeeY5AMMpkePbsngt1/umrfT70/13HyPAwPp8v9ZpAIEDLqmZ++P27qfG4qHI7OP30063QujHLwEKBIJMB3jtEpCHtgI2JRHQ5ofFebc5opKq3q+omVd20atWqRZ2jr6+Prq4uOjs76erqoq+v75TXpK/kERFqampSu3oLobu7mwsvvJDTTz+dDRs2sKKhkZirCn/MzZqzz+PoeJBAKDItCIQiMR554Sifvf9ZrvzaY9x87zP84rnheBCIxfCOH+Z/dcAPPvhWtr/rDVz2hjbqfW6uu/ZawuEwgUAAjUaYHDtKeGyYT/zN9TTXeqlOrPjp7OwkEJi+uzgftRaMMcWVSSBoVtUTyW8Sk7otSzzvUGJIiMTfw0s83pySV/qDg4PTKnPNDAb9/f3TrpBh4ZU6mQSYbHzkY39LEBfjMQ/Rqgb8ESEUg+uuvTb1mmA4yiMvHOUz9z/LO//5UW6+9xkeTHT+AnD0JVx7+1jxP9+Ax/+N+7+2jaf2PDbtPG/r2sLnt93MafVVnBzs57SV9Xz1n/7xlDufnp4eQqEQfr8fVc1qQtwYk51c9yfZyGRo6Anik8MvJ75fT7xQTUarhhI/08H0oaHtwLG0yeImVf27hY6zmDmCTIutZ1uUPRdDSbGYMhmO4g/Fx/ujMU2t0knf5ft7b72YX/Uf5+F9Izw627DPugYuOXsV3/nHT3F88OApQz6rWlq4+1v/Rq3Xhc/jwuPKfKWPFVo3Jv/yPTSdizmCPwZuBx4mXqry94HrVTWj4SER+Q7xSedmYIh4Suv/B3yPeAnMA8SXjy6Y1noxgWChyl5JC30QMyeSjx07RigUyjhwJIUiMSZDUQLhCFPh2JzVvEKRWAadf8u0pZ5dXZfS0FD/6nuNRiAS5MTRI/S/9GJW/27GmMLJ9kI0W0veWayqPxaR84HfSzz0EVU9mmkDVPW9czx1aabHWIpMawrPV884PUgkh5defPFF1q9fP+0Ysw0lJa/6J8NRJkPReev4hiIx/vX+X3Lv/7zEZN1acHlTz83V+adbu24tI4OHqfG6kGgI0Rh+v5/O9e1Z/IsZYwpttkzB2WwiXao5A4GInKOqzyWCAMArib/bRaRdVX+d/+Yt3ULF1tN1d3fPehs2MyVEciJ5cHCQhoYGxsbGGBoaYnJyktraWu574AE2d/3hvKt8kpKrfR7eN8Iv9w0TigGNZ8SfjMXg+ACXvel0rn3HW+fc4ZvM33/T1mv58Ic+xGTirsY/z3s1xpSOTC9Y82W+weKPJf7+0ix/vpjnduVM8kq/ra2N0dFR2traMhp3S5+4eeyxx1K7iJPa2toIhUIMDQ1x4MABgpEYTt8Kalvb+ZuP93LP/T87ZZVPUvqEb3K1z4PPJYJALIbrxMv4XnqQ+j078T75PZ780Z2nBAGv28nKGi/tTT7a6qupr3Zz2Tvesaj3Ot97L/SklTGVqNgLMzKuUFYKCrWzeLadv6FQiI6OjlTunwm/H091DYeOjDAZjuGtqqa5uTmVj2hVS8u0XbqZjPk/cd+/sWLyFZzRYOq55EavT37yk+z8xu0c7H+BzrVt/N3ffiwvk7bF3k9hTKXK58KMpewsvnK+A6vqPUtsW9YKFQhmTtyMjY1x4MAB3B4vq9euZ/j4CcLq4Mwzz+To0aOsXr36lMnosbEx+n7ys6wmfK+6+upTN3r5/XgcMfwnjuF1OXLeOedqEtwYU7qWEgj+NfFlC/GcQ8le4A+AR1X1slw2NBOFCgStra2pztbrq6XltHWo080rR0YAxe12s2rVKlwuF4cPH6axsZHm5mYAVJxM+Fbjbj8PWjZktNonaffu3Xz84x/HPzFBeMqPIxqirjqezygUChGJRBgaGiIYDOJyuTjzzDN56qmnFv0+Z7v6T06CNzQ0pF5nu4mNKW+LXjWkqtckDvBTYKOqDia+bwP+b47bWTJ+dP8DjE9FUHcNntpVRFV55egJVq5cic8XH/5Jv2JvbGxkdGwc7/pzkdNeS7ixE1xeIgDhaEarfZLcDmBqnNDoEYJTkwBEpjxMTU3R3NzMwYMHERGcTifRaJS9e/cuqnJZ0kKT4Em2m9iY5S2TNNTrkkEgYYj4+v9lIRyNMRmOMhWKr/D5/Fd3snJNRzwLKPFaA5FIhNHRUWpra6murgbiV/7hxvVUnfV2VjZ0EnG/utRTgPPb45u8Lj6zmQafZ87zJ/P513id7PzKF4lMjhMKxhPBORwOotEo4+PjTE1NpYIAxGsWLKWwPcy+ZK2trY0DBw7g9/sXXGVljFkeMgkEDyZyC30n8f3/AX6evybll2p8XX8gNPu6/kMHD7Fy5Uq8Xi9Hjx4lFArhdnvw+ao5/ayzeXnSTXTV2bjWvhZHIp+/AGgMjg7QEHyF9192Md2Xzp2l2+VwUON1UuN1UeV2ph7fu3cvR48eRURwOByoKrFYDIfDkRoOCofDqZVIq1evXtI649mWrLndbjZu3MjKlSttN7ExFSKTDWVbReSdwObEQ7er6g/z26zc6uvr4/O3fpmBw0OsW9/Jtdddx+bNm1PPpxdfGR8/STQaSa0ASo75s3ojh5o6cbqrSHbdGosSGtxH7PBvqZ8cxOeKl3780m8fosYt087hcjjweZ38164H+fKtX5w11XUwGExv9rTaxcmU1ECqGP3IyAgbN25c9L/LXHssbr31Vuv4jakgGS0fTeQXOktVfy4iPsCpquN5b90Mi5ks7uvr433vex8T6iUSA7fbRW1tLZ/73OfYvHnz9OIr1dUcO3aMo6MnaH7N71HVeX5qzD9JY1GCrzzP5P49BAeeRCJTuN1uzjjjjNRrkstH//2b36TG66LW66La41xwaWZraysjIyOpzj9JRKiqqiIcDuNwOHA4HMRiMWKxGK95zWuWPGFsuYSMWd5ykWvo/cTrATSp6hkichbwdVUtSIqIdIsJBOeeey579+7F09iG0+0hFosRjUY544wzuPfee1NLNqtr6gg3rCfcfCbBhg4krYxjeuc/1f8kseAE8OoVu9vt5uyzE3V1VNFoiLGjw+x//plpHXpyWepcq3+6urp4+umnOX78OKqKw+FIVSlbsWIFdXV1DA8PEwwG8Xq9tLS0EIvFZl3NU8giO8aY0paLegR/DVwEnARIVCZbahrqgtm3bx9OpxOHI/5Wk1fUAwMDhCIxDoV8xM69ghNvfj/+11xGaNU58SAQi3H6CpDf3cfI3Z/g2AP/SOC5R1JBAEiN1btcLoiGkeAEjslRgqNDdK6dvrcA4pOzY2NjvPTSS4yPjxOJRAiFQqnVPz09PbhcLtxudyrIOBwOPv7xj/Pa174Wt9vNhg0beP3rX09rayuDg4MMDg6esvs309TbxhgDmQWCoKqGkt+IiIt5CsmUolgsRjAYZGpqimAkinvt66m9+Cqu/NqjsOk9hFedAy5PvIbviZdxPdtH27N3E3rkTjxHfocjHJjjwFE06McXm2Tq+CASmSLgn5hzlc2KFSsYHBxMBZBYLEYkEsHpdKYK4CTnA1wuF9XV1axcuZI3v/nN07agnzhxggMHDhAKhVizZs0pHX2xi+wYY8pLJkNDXwBOAFcBHwJuAPaq6j/kvXUzLDYN9cDBw9S89g/wnXkBVevfgMNTnXpeAD26H9fI8/hOHiQ4fpxwOMzNN9/MTTfdxOTkFIGAf9oxNRwkFpxAw0Hq6+v5zne+k9E4e2dn56yrfFwuF+vWraOjo2PeVLTJ4Z7HHnsMh8PBaaedlkp5kf66TFNvG2Mqw5LTUAN/D1wH/Bb4APAAcEdumpd/tS1rWXfFF3B4X90EprEYrrGDfPjdb+PiM5t5eo9wx85dHD52JFUMBmBiYoJYLIaIEIuE0VCAWDAQXypKfI/B+eefP2fW0pmOHDky6+PRaJSOjo4FU9EmzzNbR5/+umJnMjTGlJd5A4GIOIFnVPUc4BuFaVJuDQ/sw33uKOKuIvjKc0z1/5rQwaepr3Jx2WeuAmDz5s3TlnoCXHX11TQ2NHD0yGFigZNEQ1Op57xeb2r8PtuNVi6Xi2j01bQTqoqqsmXLFn7zm99w+PBhqqqqUsXrZ+vAF+ros0m9bYwx884RqGoUeF5EynYnsd/v5+h9X+LIN29k9Mf/RPCFR4n4xzh27BhXXX01u3fvPuVnvG4nh17cyyqfg7XNK+KpH9JEo9HUJG42K3E2bNgAxINB+tV8a2srd911F7W1tUB8P8HLL7/M0NDQrB34QilrF5t62xhTmTKZI9gNvBH4FZAaLFfVP13yyUUGgHEgCkTmG8OC7OcI+vr6uOyyy4jFYrjqW8Hx6i5ep9PJ+vXrU/MBl1xyCbVeF3VV8d2+s2UgfeWVV4jFYlx44YWLWo6Z3NNw8uRJwuFwalloS0tLKuPnzCI33/72t2c9j63/N8ZkKhf7CC6Z7XFVfXiJbUsGgk2Zlr7MNhB0dXXx+OOPx4vK1KxExZE8Lz6fj87OTgIT46xeWc9DP3kAh+PVq/Rs8vJns2Z/tg78hhtusMldY0zeLCUNdRXwQeBM4hPFO1U1kuPGDZDHQNDZ2YnT6eTgwYO46lsIRzW1dHP9mtXUV3sgGpqzw83kqjsXhVzyXbjaGFPZlrKh7C5gE/Eg0E28RGWuKfBTEXlCRK6f7QUicr2I7BGRPSMjI1kdvLOzE7fbTXt7O263G42GiU2exB0co8ErSCw872qa7u5ufvGLX/DVr34VgBtuuOGUzVvZrtmfrQxkscvUGWMq23yBYKOq/oWq/gvwLuD383D+i1X1fOKB5q9FZPPMF6jq7aq6SVU3rVq1KquDJztYl8vF6W3NrK7z4IhMsbKpMeMOd6Fduv39/dPqE8D0pZyZHAuwyV1jTNHMNzT060QnPev3OW+IyKeACVX94lyvWWzSufThnS1btrBr1y4GBgaoq6sD4OTJk3OO7S80bJPNsI4NARljimEpcwRRXl0lJEA1EEh8raq6YokNqwEcqjqe+PpnQK+q/niun8llqcpMx/YX2qWbzRyB7fg1xhTDoucIVNWpqisSf+pU1ZX29ZKCQEIr8IiIPEV8aer98wWBXMt0bL+zs5NAYHquofR5hWzW7C90LGOMKYZMUkzkharuB+Yu45VnC6VzSMpkl26mKSZsx68xphRlkn10Wcr06jyXu3Rtx68xphRlVKGsVBRjjmC2n7OCL8aYcpKLwjTL0mKuznt7e7n88svZtWsXBw4c4NFHH+Waa66xgi/GmLJWsYEAMtswltTX18dnPvMZwuFwKmNoMBhkaGiIyy+/fM6fM8aYUlfRgQAyK+vY19fHn/3Zn8VzFs0iEolYOUhjTNmq+ECw0DLSZKCYmJiY8xiqyuHDhwmFQlYO0hhTdoq2fLRUzLeMNHknMDExwUKT6qFQiOHh4TnvGowxplRVdCDo6+vjxIkTHDp0iOrq6mlVwerq6ti6dSt+vx+3200kEplWWWwmp9NJJBIhGAwW8B0YY8zSVezQUHLIp7a2FofDQTAY5MCBA6mqYAAej4eqqipisRhut3taaoh0DoeDWCyGquL1egv5NowxZskqNhAk5wZaW1tpb29PdeATExPs2LGDkydP4vP5aG1tRVWJRqO43W4A3G43Xq8Xp9OJiOBwOHC73bS2trJx48Zivi1jjMlaxQ4Npc8N1NfXU19fn0oA193dnSoQX19fT3t7O0NDQ0xNTVFTU8MZZ5zB0NAQY2NjNDc309LSYukijDFlq2LvCBZKMZFeLGbFihWsWbOGdevW8f3vf5+nnnqKI0eOcM8993DWWWdZughjTFmzFBPzpJiwAvHGmOVgoRQTFTk0lOzgJyYmCIVCeL1eNm7ceEpHn2lWUWOMKWcVFwj6+vq45pprGB8fJxKJ4HK58Hq9drVvjKlYFTdHcNNNN3H8+HGi0ShOp5NoNMrx48e56aabit00Y4wpioq7I9i3bx8OhwOn0wmQ+nvfvn3FbJYxxhRNUe8IROSPReR5EXlRROyS3BhjiqBogUBEnMBXgW5gI/BeEcn7bqwNGzYQjUaJxWIAxGIxotEoGzZsyPepjTGmJBXzjuAtwIuqul9VQ8DdwOX5Puktt9xCc3MzDoeDcDiMw+GgubmZW265Jd+nNsaYklTMQLAGOJj2/aHEY9OIyPUiskdE9oyMjCz5pN3d3dx5551ccMEFrF27lgsuuIA777zTVgwZYypWyU8Wq+rtwO0Q31CWi2Pa/gBjjHlVMe8IDgPr0r5fm3jMGGNMARUzEPwPcJaIdIqIB3gPcG8R22OMMRWpaENDqhoRka3ATwAncKeqPlOs9hhjTKUq6hyBqj4APFDMNhhjTKWruBQTxhhjprNAYIwxFc4CQUJfXx9dXV10dnbS1dVFX19fsZtkjDEFYYGAV4vUDA4O0tTUxODgIFu3brVgYIypCBYIeLWQfU1NDSJCTU0NHo+H7du3F7tpxhiTdxYIiBey9/l80x7z+XwMDAwUp0HGGFNAFghYuJC9McYsZxYIgJ6eHkKhEH6/H1XF7/cTCoXo6ekpdtOMMSbvLBAQT0K3Y8cO2traGB0dpa2tjR07dlhiOmNMRRDVnCT0LIhNmzbpnj17it0MY4wpKyLyhKpumuv5irsjsP0CxhgzXUUFAtsvYIwxp6qoQGD7BYwx5lQVFQhsv4AxxpyqogKB7RcwxphTVVQgsP0CxhhzqqIEAhH5lIgcFpEnE3/eUYjz2n4BY4w5VTErlN2mql8s9Em7u7ut4zfGmDQVNTRkjDHmVMUMBFtF5GkRuVNEGud6kYhcLyJ7RGTPyMhIIdtnjDEVIW8pJkTk58DqWZ76B+C/gaOAAtuANlV930LHtBQTxhiTvYVSTORtjkBV35bJ60TkG8B9+WqHMcaY+RVr1VBb2rfvBH5XjHYYY4wp3qqhL4jIecSHhgaADxSpHcYYU/GKEghU9S+LcV5jjDGnsuWjxhhT4SwQGGNMhbNAYIwxFc4CgTHGVDgLBMYYU+EsEBhjTIWzQGCMMRWuYgNBX18fXV1ddHZ20tXVZQXsjTEVqyIDQV9fH1u3bmVwcJCmpiYGBwfZunWrBQNjTEWqyECwfft2PB4PNTU1iAg1NTV4PB62b99e7KYZY0zBVWQg6O/vx+fzTXvM5/MxMDBQnAYZY0wRVWQg6OzsJBAITHssEAjQ0dFRnAYZY0wRVWQg6OnpIRQK4ff7UVX8fj+hUIienp5iN80YYwquIgNBd3c3O3bsoK2tjdHRUdra2tixY4cVtTfGVKS8larMBytVaYwx2VuoVGVF3hEYY4x5lQUCY4ypcBYIjDGmwlkgMMaYCmeBwBhjKlxZrRoSkRHgQBFO3QwcLcJ5c83eR2mx91FalvP7WK+qq+b6gbIKBMUiInvmW3pVLux9lBZ7H6Wlkt+HDQ0ZY0yFs0BgjDEVzgJBZm4vdgNyxN5HabH3UVoq9n3YHIExxlQ4uyMwxpgKZ4HAGGMqnAWCBYjIgIj8VkSeFJGySX0qIneKyLCI/C7tsSYR+ZmIvJD4u7GYbczEHO/jUyJyOPGZPCki7yhmGxciIutE5CER2Ssiz4jIhxOPl9XnMc/7KLfPo0pEfiUiTyXex6cTj3eKyOMi8qKIfFdEPMVu63zmeR//V0T60z6P8xY8ls0RzE9EBoBNqlpWG01EZDMwAXxTVV+XeOwLwHFVvUVEbgIaVfXvi9nOhczxPj4FTKjqF4vZtkyJSBvQpqq/FpE64AngCuCvKKPPY5738W7K6/MQoEZVJ0TEDTwCfBj4KHCPqt4tIl8HnlLVrxWzrfOZ5318ELhPVf8j02PZHcEypaq7geMzHr4cuCvx9V3E/xOXtDneR1lR1UFV/XXi63HgWWANZfZ5zPM+yorGTSS+dSf+KNAFJDvPcvg85nofWbNAsDAFfioiT4jI9cVuzBK1qupg4usjQGsxG7NEW0Xk6cTQUUkPqaQTkQ7gjcDjlPHnMeN9QJl9HiLiFJEngWHgZ8BLwAlVjSRecogyCHIz34eqJj+PzyY+j9tExLvQcSwQLOxiVT0f6Ab+OjFUUfY0PiZYruOCXwPOAM4DBoEvFbU1GRKRWuAHwEdU9WT6c+X0eczyPsru81DVqKqeB6wF3gKcU9wWLc7M9yEirwM+Tvz9vBloAhYcbrRAsABVPZz4exj4IfFfmnI1lBjnTY73Dhe5PYuiqkOJ/wAx4BuUwWeSGMP9AfAtVb0n8XDZfR6zvY9y/DySVPUE8BBwIdAgIq7EU2uBw8VqV7bS3scfJ4bwVFWDwL+SwedhgWAeIlKTmBRDRGqAtwO/m/+nStq9wNWJr68G/rOIbVm0ZOeZ8E5K/DNJTOrtBJ5V1VvTniqrz2Ou91GGn8cqEWlIfF0N/CHx+Y6HgHclXlYOn8ds7+O5tIsLIT7PseDnYauG5iEipxO/CwBwAd9W1c8WsUkZE5HvAFuIp6QdAj4J/D/ge0A78XTe71bVkp6IneN9bCE+DKHAAPCBtLH2kiMiFwO/BH4LxBIPf4L4+HrZfB7zvI/3Ul6fxxuITwY7iV8Mf09VexP/3+8mPpzyG+AvElfVJWme9/ELYBUgwJPAB9MmlWc/lgUCY4ypbDY0ZIwxFc4CgTHGVDgLBMYYU+EsEBhjTIWzQGCMMRXOAoExGRKRK0RERaQsd6EaMxcLBMZk7r3EMzy+t9gNMSaXLBAYk4FEfp2LgWuB9yQec4jIP4vIc4l6Ag+IyLsSz71JRB5OJCv8yYzdt8aUFAsExmTmcuDHqroPOCYibwKuBDqAjcBfEs9Xk8zH8xXgXar6JuBOoCx2pJvK5Fr4JcYY4sNBX058fXfiexfw/USytSMi8lDi+bOB1wE/i6d7wUk8K6cxJckCgTELEJEm4kVLXi8iSrxjV17NQ3XKjwDPqOqFBWqiMUtiQ0PGLOxdwL+p6npV7VDVdUA/8cpp/zsxV9BKPBkewPPAKhFJDRWJyGuL0XBjMmGBwJiFvZdTr/5/AKwmXslqL/DvwK+BMVUNEQ8enxeRp4hngHxrwVprTJYs+6gxSyAitYni4SuBXwEXqeqRYrfLmGzYHIExS3NfojiIB9hmQcCUI7sjMMaYCmdzBMYYU+EsEBhjTIWzQGCMMRXOAoExxlQ4CwTGGFPh/n9jYYmjDclbkwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.regplot(x=y_train, y=y_pred, scatter_kws=dict(color='k'))\n", "plt.ylabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Summary\n", "\n", "Not bad, not bad at all.\n", "\n", "But **most importantly**\n", "- this is a more **accurate estimation** of our model's predictive efficacy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[Peer Herholz (he/him)](https://peerherholz.github.io/) \n", "Research affiliate - [NeuroDataScience lab](https://neurodatascience.github.io/) at [MNI](https://www.mcgill.ca/neuro/)/[McGill](https://www.mcgill.ca/) (& [UNIQUE](https://sites.google.com/view/unique-neuro-ai)) & [Senseable Intelligence Group](https://sensein.group/) at [McGovern Institute for Brain Research](https://mcgovern.mit.edu/)/[MIT](https://www.mit.edu/) \n", "Member - [BIDS](https://bids-specification.readthedocs.io/en/stable/), [ReproNim](https://www.repronim.org/), [Brainhack](https://brainhack.org/), [Neuromod](https://www.cneuromod.ca/), [OHBM SEA-SIG](https://ohbm-environment.org/) \n", "\n", "\"logo\" \"logo\"   @peerherholz \n", "\n", "\"logo\"" ] } ], "metadata": { "anaconda-cloud": {}, "celltoolbar": "Slideshow", "kernelspec": { "display_name": "neuro_ai", "language": "python", "name": "neuro_ai" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }