Pandas_Cookbook_103.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Beginning Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Könyv 87. oldal\n",
    "Although there is no standard approach when beginning a data analysis, it is typically a\n",
    "good idea to develop a routine for yourself when first examining a dataset.\n",
    "This\n",
    "routine can manifest itself as a dynamic checklist of tasks that evolves as your familiarity\n",
    "with pandas and data analysis expands."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exploratory Data Analysis (EDA) is a term used to encompass the entire process of\n",
    "analyzing data without the formal use of statistical testing procedures. Much of EDA\n",
    "involves visually displaying different relationships among the data to detect interesting\n",
    "patterns and develop hypotheses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>INSTNM</th>\n",
       "      <th>CITY</th>\n",
       "      <th>STABBR</th>\n",
       "      <th>HBCU</th>\n",
       "      <th>MENONLY</th>\n",
       "      <th>WOMENONLY</th>\n",
       "      <th>RELAFFIL</th>\n",
       "      <th>SATVRMID</th>\n",
       "      <th>SATMTMID</th>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <th>...</th>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <th>CURROPER</th>\n",
       "      <th>PCTPELL</th>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <th>UG25ABV</th>\n",
       "      <th>MD_EARN_WNE_P10</th>\n",
       "      <th>GRAD_DEBT_MDN_SUPP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama A &amp; M University</td>\n",
       "      <td>Normal</td>\n",
       "      <td>AL</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>424.0</td>\n",
       "      <td>420.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0059</td>\n",
       "      <td>0.0138</td>\n",
       "      <td>0.0656</td>\n",
       "      <td>1</td>\n",
       "      <td>0.7356</td>\n",
       "      <td>0.8284</td>\n",
       "      <td>0.1049</td>\n",
       "      <td>30300</td>\n",
       "      <td>33888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>University of Alabama at Birmingham</td>\n",
       "      <td>Birmingham</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>570.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0368</td>\n",
       "      <td>0.0179</td>\n",
       "      <td>0.0100</td>\n",
       "      <td>0.2607</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3460</td>\n",
       "      <td>0.5214</td>\n",
       "      <td>0.2422</td>\n",
       "      <td>39700</td>\n",
       "      <td>21941.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Amridge University</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2715</td>\n",
       "      <td>0.4536</td>\n",
       "      <td>1</td>\n",
       "      <td>0.6801</td>\n",
       "      <td>0.7795</td>\n",
       "      <td>0.8540</td>\n",
       "      <td>40100</td>\n",
       "      <td>23370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>University of Alabama in Huntsville</td>\n",
       "      <td>Huntsville</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>595.0</td>\n",
       "      <td>590.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0172</td>\n",
       "      <td>0.0332</td>\n",
       "      <td>0.0350</td>\n",
       "      <td>0.2146</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3072</td>\n",
       "      <td>0.4596</td>\n",
       "      <td>0.2640</td>\n",
       "      <td>45500</td>\n",
       "      <td>24097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alabama State University</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>AL</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>425.0</td>\n",
       "      <td>430.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0098</td>\n",
       "      <td>0.0243</td>\n",
       "      <td>0.0137</td>\n",
       "      <td>0.0892</td>\n",
       "      <td>1</td>\n",
       "      <td>0.7347</td>\n",
       "      <td>0.7554</td>\n",
       "      <td>0.1270</td>\n",
       "      <td>26600</td>\n",
       "      <td>33118.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The University of Alabama</td>\n",
       "      <td>Tuscaloosa</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>555.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0261</td>\n",
       "      <td>0.0268</td>\n",
       "      <td>0.0026</td>\n",
       "      <td>0.0844</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2040</td>\n",
       "      <td>0.4010</td>\n",
       "      <td>0.0853</td>\n",
       "      <td>41900</td>\n",
       "      <td>23750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Central Alabama Community College</td>\n",
       "      <td>Alexander City</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0019</td>\n",
       "      <td>0.3882</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5892</td>\n",
       "      <td>0.3977</td>\n",
       "      <td>0.3153</td>\n",
       "      <td>27500</td>\n",
       "      <td>16127</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>7 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                INSTNM            CITY STABBR  HBCU  MENONLY  \\\n",
       "0             Alabama A & M University          Normal     AL   1.0      0.0   \n",
       "1  University of Alabama at Birmingham      Birmingham     AL   0.0      0.0   \n",
       "2                   Amridge University      Montgomery     AL   0.0      0.0   \n",
       "3  University of Alabama in Huntsville      Huntsville     AL   0.0      0.0   \n",
       "4             Alabama State University      Montgomery     AL   1.0      0.0   \n",
       "5            The University of Alabama      Tuscaloosa     AL   0.0      0.0   \n",
       "6    Central Alabama Community College  Alexander City     AL   0.0      0.0   \n",
       "\n",
       "   WOMENONLY  RELAFFIL  SATVRMID  SATMTMID  DISTANCEONLY         ...          \\\n",
       "0        0.0         0     424.0     420.0           0.0         ...           \n",
       "1        0.0         0     570.0     565.0           0.0         ...           \n",
       "2        0.0         1       NaN       NaN           1.0         ...           \n",
       "3        0.0         0     595.0     590.0           0.0         ...           \n",
       "4        0.0         0     425.0     430.0           0.0         ...           \n",
       "5        0.0         0     555.0     565.0           0.0         ...           \n",
       "6        0.0         0       NaN       NaN           0.0         ...           \n",
       "\n",
       "   UGDS_2MOR  UGDS_NRA  UGDS_UNKN  PPTUG_EF  CURROPER  PCTPELL  PCTFLOAN  \\\n",
       "0     0.0000    0.0059     0.0138    0.0656         1   0.7356    0.8284   \n",
       "1     0.0368    0.0179     0.0100    0.2607         1   0.3460    0.5214   \n",
       "2     0.0000    0.0000     0.2715    0.4536         1   0.6801    0.7795   \n",
       "3     0.0172    0.0332     0.0350    0.2146         1   0.3072    0.4596   \n",
       "4     0.0098    0.0243     0.0137    0.0892         1   0.7347    0.7554   \n",
       "5     0.0261    0.0268     0.0026    0.0844         1   0.2040    0.4010   \n",
       "6     0.0000    0.0000     0.0019    0.3882         1   0.5892    0.3977   \n",
       "\n",
       "   UG25ABV  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP  \n",
       "0   0.1049            30300               33888  \n",
       "1   0.2422            39700             21941.5  \n",
       "2   0.8540            40100               23370  \n",
       "3   0.2640            45500               24097  \n",
       "4   0.1270            26600             33118.5  \n",
       "5   0.0853            41900               23750  \n",
       "6   0.3153            27500               16127  \n",
       "\n",
       "[7 rows x 27 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "#college = pd.read_csv('C:\\Anaconda\\data\\college.csv')\n",
    "college = pd.read_csv('https://raw.githubusercontent.com/DatasRev/source-files/master/csv/college.csv')\n",
    "college.head(7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7535, 27)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of rows and columns\n",
    "college.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 7535 entries, 0 to 7534\n",
      "Data columns (total 27 columns):\n",
      "INSTNM                7535 non-null object\n",
      "CITY                  7535 non-null object\n",
      "STABBR                7535 non-null object\n",
      "HBCU                  7164 non-null float64\n",
      "MENONLY               7164 non-null float64\n",
      "WOMENONLY             7164 non-null float64\n",
      "RELAFFIL              7535 non-null int64\n",
      "SATVRMID              1185 non-null float64\n",
      "SATMTMID              1196 non-null float64\n",
      "DISTANCEONLY          7164 non-null float64\n",
      "UGDS                  6874 non-null float64\n",
      "UGDS_WHITE            6874 non-null float64\n",
      "UGDS_BLACK            6874 non-null float64\n",
      "UGDS_HISP             6874 non-null float64\n",
      "UGDS_ASIAN            6874 non-null float64\n",
      "UGDS_AIAN             6874 non-null float64\n",
      "UGDS_NHPI             6874 non-null float64\n",
      "UGDS_2MOR             6874 non-null float64\n",
      "UGDS_NRA              6874 non-null float64\n",
      "UGDS_UNKN             6874 non-null float64\n",
      "PPTUG_EF              6853 non-null float64\n",
      "CURROPER              7535 non-null int64\n",
      "PCTPELL               6849 non-null float64\n",
      "PCTFLOAN              6849 non-null float64\n",
      "UG25ABV               6718 non-null float64\n",
      "MD_EARN_WNE_P10       6413 non-null object\n",
      "GRAD_DEBT_MDN_SUPP    7503 non-null object\n",
      "dtypes: float64(20), int64(2), object(5)\n",
      "memory usage: 1.6+ MB\n"
     ]
    }
   ],
   "source": [
    "# List the data type of each column, number of non-missing values, and memory usage with the info method:\n",
    "college.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get summary statistics for the numerical columns and transpose\n",
    "the DataFrame for more readable output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>HBCU</th>\n",
       "      <th>MENONLY</th>\n",
       "      <th>WOMENONLY</th>\n",
       "      <th>RELAFFIL</th>\n",
       "      <th>SATVRMID</th>\n",
       "      <th>SATMTMID</th>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <th>UGDS</th>\n",
       "      <th>UGDS_WHITE</th>\n",
       "      <th>UGDS_BLACK</th>\n",
       "      <th>...</th>\n",
       "      <th>UGDS_AIAN</th>\n",
       "      <th>UGDS_NHPI</th>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <th>CURROPER</th>\n",
       "      <th>PCTPELL</th>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <th>UG25ABV</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>7164.000000</td>\n",
       "      <td>7164.000000</td>\n",
       "      <td>7164.000000</td>\n",
       "      <td>7535.000000</td>\n",
       "      <td>1185.000000</td>\n",
       "      <td>1196.000000</td>\n",
       "      <td>7164.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6874.000000</td>\n",
       "      <td>6853.000000</td>\n",
       "      <td>7535.000000</td>\n",
       "      <td>6849.000000</td>\n",
       "      <td>6849.000000</td>\n",
       "      <td>6718.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.014238</td>\n",
       "      <td>0.009213</td>\n",
       "      <td>0.005304</td>\n",
       "      <td>0.190975</td>\n",
       "      <td>522.819409</td>\n",
       "      <td>530.765050</td>\n",
       "      <td>0.005583</td>\n",
       "      <td>2356.837940</td>\n",
       "      <td>0.510207</td>\n",
       "      <td>0.189997</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013813</td>\n",
       "      <td>0.004569</td>\n",
       "      <td>0.023950</td>\n",
       "      <td>0.016086</td>\n",
       "      <td>0.045181</td>\n",
       "      <td>0.226639</td>\n",
       "      <td>0.923291</td>\n",
       "      <td>0.530643</td>\n",
       "      <td>0.522211</td>\n",
       "      <td>0.410021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.118478</td>\n",
       "      <td>0.095546</td>\n",
       "      <td>0.072642</td>\n",
       "      <td>0.393096</td>\n",
       "      <td>68.578862</td>\n",
       "      <td>73.469767</td>\n",
       "      <td>0.074519</td>\n",
       "      <td>5474.275871</td>\n",
       "      <td>0.286958</td>\n",
       "      <td>0.224587</td>\n",
       "      <td>...</td>\n",
       "      <td>0.070196</td>\n",
       "      <td>0.033125</td>\n",
       "      <td>0.031288</td>\n",
       "      <td>0.050172</td>\n",
       "      <td>0.093440</td>\n",
       "      <td>0.246470</td>\n",
       "      <td>0.266146</td>\n",
       "      <td>0.225544</td>\n",
       "      <td>0.283616</td>\n",
       "      <td>0.228939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>290.000000</td>\n",
       "      <td>310.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>475.000000</td>\n",
       "      <td>482.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>0.267500</td>\n",
       "      <td>0.036125</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.357800</td>\n",
       "      <td>0.332900</td>\n",
       "      <td>0.241500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>510.000000</td>\n",
       "      <td>520.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>412.500000</td>\n",
       "      <td>0.555700</td>\n",
       "      <td>0.100050</td>\n",
       "      <td>...</td>\n",
       "      <td>0.002600</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.017500</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.014300</td>\n",
       "      <td>0.150400</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.521500</td>\n",
       "      <td>0.583300</td>\n",
       "      <td>0.400750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>555.000000</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1929.500000</td>\n",
       "      <td>0.747875</td>\n",
       "      <td>0.257700</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007300</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.033900</td>\n",
       "      <td>0.011700</td>\n",
       "      <td>0.045400</td>\n",
       "      <td>0.376900</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.712900</td>\n",
       "      <td>0.745000</td>\n",
       "      <td>0.572275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>765.000000</td>\n",
       "      <td>785.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>151558.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.998300</td>\n",
       "      <td>0.533300</td>\n",
       "      <td>0.928600</td>\n",
       "      <td>0.902700</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              HBCU      MENONLY    WOMENONLY     RELAFFIL     SATVRMID  \\\n",
       "count  7164.000000  7164.000000  7164.000000  7535.000000  1185.000000   \n",
       "mean      0.014238     0.009213     0.005304     0.190975   522.819409   \n",
       "std       0.118478     0.095546     0.072642     0.393096    68.578862   \n",
       "min       0.000000     0.000000     0.000000     0.000000   290.000000   \n",
       "25%       0.000000     0.000000     0.000000     0.000000   475.000000   \n",
       "50%       0.000000     0.000000     0.000000     0.000000   510.000000   \n",
       "75%       0.000000     0.000000     0.000000     0.000000   555.000000   \n",
       "max       1.000000     1.000000     1.000000     1.000000   765.000000   \n",
       "\n",
       "          SATMTMID  DISTANCEONLY           UGDS   UGDS_WHITE   UGDS_BLACK  \\\n",
       "count  1196.000000   7164.000000    6874.000000  6874.000000  6874.000000   \n",
       "mean    530.765050      0.005583    2356.837940     0.510207     0.189997   \n",
       "std      73.469767      0.074519    5474.275871     0.286958     0.224587   \n",
       "min     310.000000      0.000000       0.000000     0.000000     0.000000   \n",
       "25%     482.000000      0.000000     117.000000     0.267500     0.036125   \n",
       "50%     520.000000      0.000000     412.500000     0.555700     0.100050   \n",
       "75%     565.000000      0.000000    1929.500000     0.747875     0.257700   \n",
       "max     785.000000      1.000000  151558.000000     1.000000     1.000000   \n",
       "\n",
       "          ...         UGDS_AIAN    UGDS_NHPI    UGDS_2MOR     UGDS_NRA  \\\n",
       "count     ...       6874.000000  6874.000000  6874.000000  6874.000000   \n",
       "mean      ...          0.013813     0.004569     0.023950     0.016086   \n",
       "std       ...          0.070196     0.033125     0.031288     0.050172   \n",
       "min       ...          0.000000     0.000000     0.000000     0.000000   \n",
       "25%       ...          0.000000     0.000000     0.000000     0.000000   \n",
       "50%       ...          0.002600     0.000000     0.017500     0.000000   \n",
       "75%       ...          0.007300     0.002500     0.033900     0.011700   \n",
       "max       ...          1.000000     0.998300     0.533300     0.928600   \n",
       "\n",
       "         UGDS_UNKN     PPTUG_EF     CURROPER      PCTPELL     PCTFLOAN  \\\n",
       "count  6874.000000  6853.000000  7535.000000  6849.000000  6849.000000   \n",
       "mean      0.045181     0.226639     0.923291     0.530643     0.522211   \n",
       "std       0.093440     0.246470     0.266146     0.225544     0.283616   \n",
       "min       0.000000     0.000000     0.000000     0.000000     0.000000   \n",
       "25%       0.000000     0.000000     1.000000     0.357800     0.332900   \n",
       "50%       0.014300     0.150400     1.000000     0.521500     0.583300   \n",
       "75%       0.045400     0.376900     1.000000     0.712900     0.745000   \n",
       "max       0.902700     1.000000     1.000000     1.000000     1.000000   \n",
       "\n",
       "           UG25ABV  \n",
       "count  6718.000000  \n",
       "mean      0.410021  \n",
       "std       0.228939  \n",
       "min       0.000000  \n",
       "25%       0.241500  \n",
       "50%       0.400750  \n",
       "75%       0.572275  \n",
       "max       1.000000  \n",
       "\n",
       "[8 rows x 22 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "college.describe(include=[np.number])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>HBCU</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.014238</td>\n",
       "      <td>0.118478</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.009213</td>\n",
       "      <td>0.095546</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WOMENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005304</td>\n",
       "      <td>0.072642</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RELAFFIL</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.190975</td>\n",
       "      <td>0.393096</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATVRMID</th>\n",
       "      <td>1185.0</td>\n",
       "      <td>522.819409</td>\n",
       "      <td>68.578862</td>\n",
       "      <td>290.0</td>\n",
       "      <td>475.000000</td>\n",
       "      <td>510.00000</td>\n",
       "      <td>555.000000</td>\n",
       "      <td>765.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATMTMID</th>\n",
       "      <td>1196.0</td>\n",
       "      <td>530.765050</td>\n",
       "      <td>73.469767</td>\n",
       "      <td>310.0</td>\n",
       "      <td>482.000000</td>\n",
       "      <td>520.00000</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>785.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005583</td>\n",
       "      <td>0.074519</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>2356.837940</td>\n",
       "      <td>5474.275871</td>\n",
       "      <td>0.0</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>412.50000</td>\n",
       "      <td>1929.500000</td>\n",
       "      <td>151558.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_WHITE</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.510207</td>\n",
       "      <td>0.286958</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.267500</td>\n",
       "      <td>0.55570</td>\n",
       "      <td>0.747875</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_BLACK</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.189997</td>\n",
       "      <td>0.224587</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.036125</td>\n",
       "      <td>0.10005</td>\n",
       "      <td>0.257700</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_HISP</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.161635</td>\n",
       "      <td>0.221854</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.027600</td>\n",
       "      <td>0.07140</td>\n",
       "      <td>0.198875</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_ASIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.033544</td>\n",
       "      <td>0.073777</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.01290</td>\n",
       "      <td>0.032700</td>\n",
       "      <td>0.9727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_AIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.013813</td>\n",
       "      <td>0.070196</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00260</td>\n",
       "      <td>0.007300</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NHPI</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.004569</td>\n",
       "      <td>0.033125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.9983</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.023950</td>\n",
       "      <td>0.031288</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01750</td>\n",
       "      <td>0.033900</td>\n",
       "      <td>0.5333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.016086</td>\n",
       "      <td>0.050172</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.011700</td>\n",
       "      <td>0.9286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.045181</td>\n",
       "      <td>0.093440</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01430</td>\n",
       "      <td>0.045400</td>\n",
       "      <td>0.9027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <td>6853.0</td>\n",
       "      <td>0.226639</td>\n",
       "      <td>0.246470</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.15040</td>\n",
       "      <td>0.376900</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CURROPER</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.923291</td>\n",
       "      <td>0.266146</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTPELL</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.530643</td>\n",
       "      <td>0.225544</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.357800</td>\n",
       "      <td>0.52150</td>\n",
       "      <td>0.712900</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.522211</td>\n",
       "      <td>0.283616</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.332900</td>\n",
       "      <td>0.58330</td>\n",
       "      <td>0.745000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UG25ABV</th>\n",
       "      <td>6718.0</td>\n",
       "      <td>0.410021</td>\n",
       "      <td>0.228939</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.241500</td>\n",
       "      <td>0.40075</td>\n",
       "      <td>0.572275</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               count         mean          std    min         25%        50%  \\\n",
       "HBCU          7164.0     0.014238     0.118478    0.0    0.000000    0.00000   \n",
       "MENONLY       7164.0     0.009213     0.095546    0.0    0.000000    0.00000   \n",
       "WOMENONLY     7164.0     0.005304     0.072642    0.0    0.000000    0.00000   \n",
       "RELAFFIL      7535.0     0.190975     0.393096    0.0    0.000000    0.00000   \n",
       "SATVRMID      1185.0   522.819409    68.578862  290.0  475.000000  510.00000   \n",
       "SATMTMID      1196.0   530.765050    73.469767  310.0  482.000000  520.00000   \n",
       "DISTANCEONLY  7164.0     0.005583     0.074519    0.0    0.000000    0.00000   \n",
       "UGDS          6874.0  2356.837940  5474.275871    0.0  117.000000  412.50000   \n",
       "UGDS_WHITE    6874.0     0.510207     0.286958    0.0    0.267500    0.55570   \n",
       "UGDS_BLACK    6874.0     0.189997     0.224587    0.0    0.036125    0.10005   \n",
       "UGDS_HISP     6874.0     0.161635     0.221854    0.0    0.027600    0.07140   \n",
       "UGDS_ASIAN    6874.0     0.033544     0.073777    0.0    0.002500    0.01290   \n",
       "UGDS_AIAN     6874.0     0.013813     0.070196    0.0    0.000000    0.00260   \n",
       "UGDS_NHPI     6874.0     0.004569     0.033125    0.0    0.000000    0.00000   \n",
       "UGDS_2MOR     6874.0     0.023950     0.031288    0.0    0.000000    0.01750   \n",
       "UGDS_NRA      6874.0     0.016086     0.050172    0.0    0.000000    0.00000   \n",
       "UGDS_UNKN     6874.0     0.045181     0.093440    0.0    0.000000    0.01430   \n",
       "PPTUG_EF      6853.0     0.226639     0.246470    0.0    0.000000    0.15040   \n",
       "CURROPER      7535.0     0.923291     0.266146    0.0    1.000000    1.00000   \n",
       "PCTPELL       6849.0     0.530643     0.225544    0.0    0.357800    0.52150   \n",
       "PCTFLOAN      6849.0     0.522211     0.283616    0.0    0.332900    0.58330   \n",
       "UG25ABV       6718.0     0.410021     0.228939    0.0    0.241500    0.40075   \n",
       "\n",
       "                      75%          max  \n",
       "HBCU             0.000000       1.0000  \n",
       "MENONLY          0.000000       1.0000  \n",
       "WOMENONLY        0.000000       1.0000  \n",
       "RELAFFIL         0.000000       1.0000  \n",
       "SATVRMID       555.000000     765.0000  \n",
       "SATMTMID       565.000000     785.0000  \n",
       "DISTANCEONLY     0.000000       1.0000  \n",
       "UGDS          1929.500000  151558.0000  \n",
       "UGDS_WHITE       0.747875       1.0000  \n",
       "UGDS_BLACK       0.257700       1.0000  \n",
       "UGDS_HISP        0.198875       1.0000  \n",
       "UGDS_ASIAN       0.032700       0.9727  \n",
       "UGDS_AIAN        0.007300       1.0000  \n",
       "UGDS_NHPI        0.002500       0.9983  \n",
       "UGDS_2MOR        0.033900       0.5333  \n",
       "UGDS_NRA         0.011700       0.9286  \n",
       "UGDS_UNKN        0.045400       0.9027  \n",
       "PPTUG_EF         0.376900       1.0000  \n",
       "CURROPER         1.000000       1.0000  \n",
       "PCTPELL          0.712900       1.0000  \n",
       "PCTFLOAN         0.745000       1.0000  \n",
       "UG25ABV          0.572275       1.0000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "college.describe(include=[np.number]).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>unique</th>\n",
       "      <th>top</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>INSTNM</th>\n",
       "      <td>7535</td>\n",
       "      <td>7535</td>\n",
       "      <td>Dance Theatre of Harlem Inc</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CITY</th>\n",
       "      <td>7535</td>\n",
       "      <td>2514</td>\n",
       "      <td>New York</td>\n",
       "      <td>87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>STABBR</th>\n",
       "      <td>7535</td>\n",
       "      <td>59</td>\n",
       "      <td>CA</td>\n",
       "      <td>773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MD_EARN_WNE_P10</th>\n",
       "      <td>6413</td>\n",
       "      <td>598</td>\n",
       "      <td>PrivacySuppressed</td>\n",
       "      <td>822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GRAD_DEBT_MDN_SUPP</th>\n",
       "      <td>7503</td>\n",
       "      <td>2038</td>\n",
       "      <td>PrivacySuppressed</td>\n",
       "      <td>1510</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   count unique                          top  freq\n",
       "INSTNM              7535   7535  Dance Theatre of Harlem Inc     1\n",
       "CITY                7535   2514                     New York    87\n",
       "STABBR              7535     59                           CA   773\n",
       "MD_EARN_WNE_P10     6413    598            PrivacySuppressed   822\n",
       "GRAD_DEBT_MDN_SUPP  7503   2038            PrivacySuppressed  1510"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get summary statistics for the object and categorical columns:\n",
    "college.describe(include=[np.object, pd.Categorical]).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Broadly speaking, we can classify data as being either continuous or\n",
    "categorical. Continuous data is always numeric and can usually take on an\n",
    "infinite number of possibilities such as height, weight, and salary.\n",
    "Categorical data represents discrete values that take on a finite number of\n",
    "possibilities such as ethnicity, employment status, and car color.\n",
    "Categorical data can be represented numerically or with characters.\n",
    "Categorical columns are usually going to be either of type np.object or pd.Categorical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>1%</th>\n",
       "      <th>5%</th>\n",
       "      <th>10%</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>90%</th>\n",
       "      <th>95%</th>\n",
       "      <th>99%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>HBCU</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.014238</td>\n",
       "      <td>0.118478</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.009213</td>\n",
       "      <td>0.095546</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WOMENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005304</td>\n",
       "      <td>0.072642</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RELAFFIL</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.190975</td>\n",
       "      <td>0.393096</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATVRMID</th>\n",
       "      <td>1185.0</td>\n",
       "      <td>522.819409</td>\n",
       "      <td>68.578862</td>\n",
       "      <td>290.0</td>\n",
       "      <td>390.000000</td>\n",
       "      <td>430.000000</td>\n",
       "      <td>447.40000</td>\n",
       "      <td>475.000000</td>\n",
       "      <td>510.00000</td>\n",
       "      <td>555.000000</td>\n",
       "      <td>605.00000</td>\n",
       "      <td>665.000000</td>\n",
       "      <td>730.000000</td>\n",
       "      <td>765.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATMTMID</th>\n",
       "      <td>1196.0</td>\n",
       "      <td>530.765050</td>\n",
       "      <td>73.469767</td>\n",
       "      <td>310.0</td>\n",
       "      <td>395.000000</td>\n",
       "      <td>430.000000</td>\n",
       "      <td>453.00000</td>\n",
       "      <td>482.000000</td>\n",
       "      <td>520.00000</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>630.00000</td>\n",
       "      <td>685.000000</td>\n",
       "      <td>745.250000</td>\n",
       "      <td>785.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005583</td>\n",
       "      <td>0.074519</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>2356.837940</td>\n",
       "      <td>5474.275871</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>31.650000</td>\n",
       "      <td>49.00000</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>412.50000</td>\n",
       "      <td>1929.500000</td>\n",
       "      <td>6512.30000</td>\n",
       "      <td>11858.050000</td>\n",
       "      <td>26015.290000</td>\n",
       "      <td>151558.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_WHITE</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.510207</td>\n",
       "      <td>0.286958</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.013265</td>\n",
       "      <td>0.06879</td>\n",
       "      <td>0.267500</td>\n",
       "      <td>0.55570</td>\n",
       "      <td>0.747875</td>\n",
       "      <td>0.86297</td>\n",
       "      <td>0.927315</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_BLACK</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.189997</td>\n",
       "      <td>0.224587</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00753</td>\n",
       "      <td>0.036125</td>\n",
       "      <td>0.10005</td>\n",
       "      <td>0.257700</td>\n",
       "      <td>0.51571</td>\n",
       "      <td>0.726715</td>\n",
       "      <td>0.961467</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_HISP</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.161635</td>\n",
       "      <td>0.221854</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00573</td>\n",
       "      <td>0.027600</td>\n",
       "      <td>0.07140</td>\n",
       "      <td>0.198875</td>\n",
       "      <td>0.44314</td>\n",
       "      <td>0.666700</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_ASIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.033544</td>\n",
       "      <td>0.073777</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.01290</td>\n",
       "      <td>0.032700</td>\n",
       "      <td>0.07840</td>\n",
       "      <td>0.131775</td>\n",
       "      <td>0.346429</td>\n",
       "      <td>0.9727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_AIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.013813</td>\n",
       "      <td>0.070196</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00260</td>\n",
       "      <td>0.007300</td>\n",
       "      <td>0.01837</td>\n",
       "      <td>0.036410</td>\n",
       "      <td>0.209326</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NHPI</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.004569</td>\n",
       "      <td>0.033125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.00800</td>\n",
       "      <td>0.015200</td>\n",
       "      <td>0.050508</td>\n",
       "      <td>0.9983</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.023950</td>\n",
       "      <td>0.031288</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01750</td>\n",
       "      <td>0.033900</td>\n",
       "      <td>0.05651</td>\n",
       "      <td>0.076900</td>\n",
       "      <td>0.133154</td>\n",
       "      <td>0.5333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.016086</td>\n",
       "      <td>0.050172</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.011700</td>\n",
       "      <td>0.04250</td>\n",
       "      <td>0.078135</td>\n",
       "      <td>0.236989</td>\n",
       "      <td>0.9286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.045181</td>\n",
       "      <td>0.093440</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01430</td>\n",
       "      <td>0.045400</td>\n",
       "      <td>0.10801</td>\n",
       "      <td>0.200575</td>\n",
       "      <td>0.496581</td>\n",
       "      <td>0.9027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <td>6853.0</td>\n",
       "      <td>0.226639</td>\n",
       "      <td>0.246470</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.15040</td>\n",
       "      <td>0.376900</td>\n",
       "      <td>0.60410</td>\n",
       "      <td>0.710620</td>\n",
       "      <td>0.946724</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CURROPER</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.923291</td>\n",
       "      <td>0.266146</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTPELL</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.530643</td>\n",
       "      <td>0.225544</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.054976</td>\n",
       "      <td>0.170000</td>\n",
       "      <td>0.23656</td>\n",
       "      <td>0.357800</td>\n",
       "      <td>0.52150</td>\n",
       "      <td>0.712900</td>\n",
       "      <td>0.83330</td>\n",
       "      <td>0.896360</td>\n",
       "      <td>0.993908</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.522211</td>\n",
       "      <td>0.283616</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.332900</td>\n",
       "      <td>0.58330</td>\n",
       "      <td>0.745000</td>\n",
       "      <td>0.84752</td>\n",
       "      <td>0.897920</td>\n",
       "      <td>0.986368</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UG25ABV</th>\n",
       "      <td>6718.0</td>\n",
       "      <td>0.410021</td>\n",
       "      <td>0.228939</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.037400</td>\n",
       "      <td>0.08990</td>\n",
       "      <td>0.241500</td>\n",
       "      <td>0.40075</td>\n",
       "      <td>0.572275</td>\n",
       "      <td>0.72666</td>\n",
       "      <td>0.800000</td>\n",
       "      <td>0.917383</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               count         mean          std    min          1%          5%  \\\n",
       "HBCU          7164.0     0.014238     0.118478    0.0    0.000000    0.000000   \n",
       "MENONLY       7164.0     0.009213     0.095546    0.0    0.000000    0.000000   \n",
       "WOMENONLY     7164.0     0.005304     0.072642    0.0    0.000000    0.000000   \n",
       "RELAFFIL      7535.0     0.190975     0.393096    0.0    0.000000    0.000000   \n",
       "SATVRMID      1185.0   522.819409    68.578862  290.0  390.000000  430.000000   \n",
       "SATMTMID      1196.0   530.765050    73.469767  310.0  395.000000  430.000000   \n",
       "DISTANCEONLY  7164.0     0.005583     0.074519    0.0    0.000000    0.000000   \n",
       "UGDS          6874.0  2356.837940  5474.275871    0.0   14.000000   31.650000   \n",
       "UGDS_WHITE    6874.0     0.510207     0.286958    0.0    0.000000    0.013265   \n",
       "UGDS_BLACK    6874.0     0.189997     0.224587    0.0    0.000000    0.000000   \n",
       "UGDS_HISP     6874.0     0.161635     0.221854    0.0    0.000000    0.000000   \n",
       "UGDS_ASIAN    6874.0     0.033544     0.073777    0.0    0.000000    0.000000   \n",
       "UGDS_AIAN     6874.0     0.013813     0.070196    0.0    0.000000    0.000000   \n",
       "UGDS_NHPI     6874.0     0.004569     0.033125    0.0    0.000000    0.000000   \n",
       "UGDS_2MOR     6874.0     0.023950     0.031288    0.0    0.000000    0.000000   \n",
       "UGDS_NRA      6874.0     0.016086     0.050172    0.0    0.000000    0.000000   \n",
       "UGDS_UNKN     6874.0     0.045181     0.093440    0.0    0.000000    0.000000   \n",
       "PPTUG_EF      6853.0     0.226639     0.246470    0.0    0.000000    0.000000   \n",
       "CURROPER      7535.0     0.923291     0.266146    0.0    0.000000    0.000000   \n",
       "PCTPELL       6849.0     0.530643     0.225544    0.0    0.054976    0.170000   \n",
       "PCTFLOAN      6849.0     0.522211     0.283616    0.0    0.000000    0.000000   \n",
       "UG25ABV       6718.0     0.410021     0.228939    0.0    0.002500    0.037400   \n",
       "\n",
       "                    10%         25%        50%          75%         90%  \\\n",
       "HBCU            0.00000    0.000000    0.00000     0.000000     0.00000   \n",
       "MENONLY         0.00000    0.000000    0.00000     0.000000     0.00000   \n",
       "WOMENONLY       0.00000    0.000000    0.00000     0.000000     0.00000   \n",
       "RELAFFIL        0.00000    0.000000    0.00000     0.000000     1.00000   \n",
       "SATVRMID      447.40000  475.000000  510.00000   555.000000   605.00000   \n",
       "SATMTMID      453.00000  482.000000  520.00000   565.000000   630.00000   \n",
       "DISTANCEONLY    0.00000    0.000000    0.00000     0.000000     0.00000   \n",
       "UGDS           49.00000  117.000000  412.50000  1929.500000  6512.30000   \n",
       "UGDS_WHITE      0.06879    0.267500    0.55570     0.747875     0.86297   \n",
       "UGDS_BLACK      0.00753    0.036125    0.10005     0.257700     0.51571   \n",
       "UGDS_HISP       0.00573    0.027600    0.07140     0.198875     0.44314   \n",
       "UGDS_ASIAN      0.00000    0.002500    0.01290     0.032700     0.07840   \n",
       "UGDS_AIAN       0.00000    0.000000    0.00260     0.007300     0.01837   \n",
       "UGDS_NHPI       0.00000    0.000000    0.00000     0.002500     0.00800   \n",
       "UGDS_2MOR       0.00000    0.000000    0.01750     0.033900     0.05651   \n",
       "UGDS_NRA        0.00000    0.000000    0.00000     0.011700     0.04250   \n",
       "UGDS_UNKN       0.00000    0.000000    0.01430     0.045400     0.10801   \n",
       "PPTUG_EF        0.00000    0.000000    0.15040     0.376900     0.60410   \n",
       "CURROPER        1.00000    1.000000    1.00000     1.000000     1.00000   \n",
       "PCTPELL         0.23656    0.357800    0.52150     0.712900     0.83330   \n",
       "PCTFLOAN        0.00000    0.332900    0.58330     0.745000     0.84752   \n",
       "UG25ABV         0.08990    0.241500    0.40075     0.572275     0.72666   \n",
       "\n",
       "                       95%           99%          max  \n",
       "HBCU              0.000000      1.000000       1.0000  \n",
       "MENONLY           0.000000      0.000000       1.0000  \n",
       "WOMENONLY         0.000000      0.000000       1.0000  \n",
       "RELAFFIL          1.000000      1.000000       1.0000  \n",
       "SATVRMID        665.000000    730.000000     765.0000  \n",
       "SATMTMID        685.000000    745.250000     785.0000  \n",
       "DISTANCEONLY      0.000000      0.000000       1.0000  \n",
       "UGDS          11858.050000  26015.290000  151558.0000  \n",
       "UGDS_WHITE        0.927315      1.000000       1.0000  \n",
       "UGDS_BLACK        0.726715      0.961467       1.0000  \n",
       "UGDS_HISP         0.666700      1.000000       1.0000  \n",
       "UGDS_ASIAN        0.131775      0.346429       0.9727  \n",
       "UGDS_AIAN         0.036410      0.209326       1.0000  \n",
       "UGDS_NHPI         0.015200      0.050508       0.9983  \n",
       "UGDS_2MOR         0.076900      0.133154       0.5333  \n",
       "UGDS_NRA          0.078135      0.236989       0.9286  \n",
       "UGDS_UNKN         0.200575      0.496581       0.9027  \n",
       "PPTUG_EF          0.710620      0.946724       1.0000  \n",
       "CURROPER          1.000000      1.000000       1.0000  \n",
       "PCTPELL           0.896360      0.993908       1.0000  \n",
       "PCTFLOAN          0.897920      0.986368       1.0000  \n",
       "UG25ABV           0.800000      0.917383       1.0000  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "college.describe(include=[np.number],\n",
    "percentiles=[.01, .05, .10, .25, .5,\n",
    ".75, .9, .95, .99]).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data dictionaries\n",
    "A crucial part of a data analysis involves creating and maintaining a data dictionary. A data\n",
    "dictionary is a table of metadata and notes on each column of data. One of the primary\n",
    "purposes of a data dictionary is to explain the meaning of the column names. The college\n",
    "dataset uses a lot of abbreviations that are likely to be unfamiliar to an analyst who is\n",
    "inspecting it for the first time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>INSTNM</th>\n",
       "      <th>CITY</th>\n",
       "      <th>STABBR</th>\n",
       "      <th>HBCU</th>\n",
       "      <th>MENONLY</th>\n",
       "      <th>WOMENONLY</th>\n",
       "      <th>RELAFFIL</th>\n",
       "      <th>SATVRMID</th>\n",
       "      <th>SATMTMID</th>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <th>...</th>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <th>CURROPER</th>\n",
       "      <th>PCTPELL</th>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <th>UG25ABV</th>\n",
       "      <th>MD_EARN_WNE_P10</th>\n",
       "      <th>GRAD_DEBT_MDN_SUPP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama A &amp; M University</td>\n",
       "      <td>Normal</td>\n",
       "      <td>AL</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>424.0</td>\n",
       "      <td>420.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0059</td>\n",
       "      <td>0.0138</td>\n",
       "      <td>0.0656</td>\n",
       "      <td>1</td>\n",
       "      <td>0.7356</td>\n",
       "      <td>0.8284</td>\n",
       "      <td>0.1049</td>\n",
       "      <td>30300</td>\n",
       "      <td>33888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>University of Alabama at Birmingham</td>\n",
       "      <td>Birmingham</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>570.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0368</td>\n",
       "      <td>0.0179</td>\n",
       "      <td>0.0100</td>\n",
       "      <td>0.2607</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3460</td>\n",
       "      <td>0.5214</td>\n",
       "      <td>0.2422</td>\n",
       "      <td>39700</td>\n",
       "      <td>21941.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Amridge University</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2715</td>\n",
       "      <td>0.4536</td>\n",
       "      <td>1</td>\n",
       "      <td>0.6801</td>\n",
       "      <td>0.7795</td>\n",
       "      <td>0.8540</td>\n",
       "      <td>40100</td>\n",
       "      <td>23370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>University of Alabama in Huntsville</td>\n",
       "      <td>Huntsville</td>\n",
       "      <td>AL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>595.0</td>\n",
       "      <td>590.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0172</td>\n",
       "      <td>0.0332</td>\n",
       "      <td>0.0350</td>\n",
       "      <td>0.2146</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3072</td>\n",
       "      <td>0.4596</td>\n",
       "      <td>0.2640</td>\n",
       "      <td>45500</td>\n",
       "      <td>24097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alabama State University</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>AL</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>425.0</td>\n",
       "      <td>430.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0098</td>\n",
       "      <td>0.0243</td>\n",
       "      <td>0.0137</td>\n",
       "      <td>0.0892</td>\n",
       "      <td>1</td>\n",
       "      <td>0.7347</td>\n",
       "      <td>0.7554</td>\n",
       "      <td>0.1270</td>\n",
       "      <td>26600</td>\n",
       "      <td>33118.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                INSTNM        CITY STABBR  HBCU  MENONLY  \\\n",
       "0             Alabama A & M University      Normal     AL   1.0      0.0   \n",
       "1  University of Alabama at Birmingham  Birmingham     AL   0.0      0.0   \n",
       "2                   Amridge University  Montgomery     AL   0.0      0.0   \n",
       "3  University of Alabama in Huntsville  Huntsville     AL   0.0      0.0   \n",
       "4             Alabama State University  Montgomery     AL   1.0      0.0   \n",
       "\n",
       "   WOMENONLY  RELAFFIL  SATVRMID  SATMTMID  DISTANCEONLY         ...          \\\n",
       "0        0.0         0     424.0     420.0           0.0         ...           \n",
       "1        0.0         0     570.0     565.0           0.0         ...           \n",
       "2        0.0         1       NaN       NaN           1.0         ...           \n",
       "3        0.0         0     595.0     590.0           0.0         ...           \n",
       "4        0.0         0     425.0     430.0           0.0         ...           \n",
       "\n",
       "   UGDS_2MOR  UGDS_NRA  UGDS_UNKN  PPTUG_EF  CURROPER  PCTPELL  PCTFLOAN  \\\n",
       "0     0.0000    0.0059     0.0138    0.0656         1   0.7356    0.8284   \n",
       "1     0.0368    0.0179     0.0100    0.2607         1   0.3460    0.5214   \n",
       "2     0.0000    0.0000     0.2715    0.4536         1   0.6801    0.7795   \n",
       "3     0.0172    0.0332     0.0350    0.2146         1   0.3072    0.4596   \n",
       "4     0.0098    0.0243     0.0137    0.0892         1   0.7347    0.7554   \n",
       "\n",
       "   UG25ABV  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP  \n",
       "0   0.1049            30300               33888  \n",
       "1   0.2422            39700             21941.5  \n",
       "2   0.8540            40100               23370  \n",
       "3   0.2640            45500               24097  \n",
       "4   0.1270            26600             33118.5  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collegedict = pd.read_csv('https://raw.githubusercontent.com/DatasRev/source-files/master/csv/college.csv')\n",
    "collegedict.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reducing memory by changing data types\n",
    "This recipe changes the data type of one of the object columns from the college dataset to\n",
    "the special pandas Categorical data type to drastically reduce its memory usage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>RELAFFIL</th>\n",
       "      <th>SATMTMID</th>\n",
       "      <th>CURROPER</th>\n",
       "      <th>INSTNM</th>\n",
       "      <th>STABBR</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>420.0</td>\n",
       "      <td>1</td>\n",
       "      <td>Alabama A &amp; M University</td>\n",
       "      <td>AL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>1</td>\n",
       "      <td>University of Alabama at Birmingham</td>\n",
       "      <td>AL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>Amridge University</td>\n",
       "      <td>AL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>590.0</td>\n",
       "      <td>1</td>\n",
       "      <td>University of Alabama in Huntsville</td>\n",
       "      <td>AL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>430.0</td>\n",
       "      <td>1</td>\n",
       "      <td>Alabama State University</td>\n",
       "      <td>AL</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   RELAFFIL  SATMTMID  CURROPER                               INSTNM STABBR\n",
       "0         0     420.0         1             Alabama A & M University     AL\n",
       "1         0     565.0         1  University of Alabama at Birmingham     AL\n",
       "2         1       NaN         1                   Amridge University     AL\n",
       "3         0     590.0         1  University of Alabama in Huntsville     AL\n",
       "4         0     430.0         1             Alabama State University     AL"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "different_cols = ['RELAFFIL', 'SATMTMID', 'CURROPER',\n",
    "'INSTNM', 'STABBR']\n",
    "col2 = college.loc[:, different_cols]\n",
    "col2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RELAFFIL      int64\n",
       "SATMTMID    float64\n",
       "CURROPER      int64\n",
       "INSTNM       object\n",
       "STABBR       object\n",
       "dtype: object"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index           80\n",
       "RELAFFIL     60280\n",
       "SATMTMID     60280\n",
       "CURROPER     60280\n",
       "INSTNM      660240\n",
       "STABBR      444565\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the memory usage of each column with the memory_usage method:\n",
    "original_mem = col2.memory_usage(deep=True)\n",
    "original_mem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is no need to use 64 bits for the RELAFFIL column as it contains only 0/1\n",
    "values. Let's convert this column to an 8-bit (1 byte) integer with the astype\n",
    "method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RELAFFIL       int8\n",
       "SATMTMID    float64\n",
       "CURROPER      int64\n",
       "INSTNM       object\n",
       "STABBR       object\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col2['RELAFFIL'] = col2['RELAFFIL'].astype(np.int8)\n",
    "col2.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Find the memory usage of each column again and note the large reduction:\n",
    "#http://pandas.pydata.org/pandas-docs/version/0.19/generated/pandas.DataFrame.memory_usage.html\n",
    "Pandas dataframe.memory_usage() function return the memory usage of each column in bytes. The memory usage can optionally include\n",
    "the contribution of the index and elements of object dtype."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index           80\n",
       "RELAFFIL     60280\n",
       "SATMTMID     60280\n",
       "CURROPER     60280\n",
       "INSTNM      660240\n",
       "STABBR      444565\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "college[different_cols].memory_usage(deep=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To save even more memory, you will want to consider changing object data types\n",
    "to categorical if they have a reasonably low cardinality (number of unique\n",
    "values). Let's first check the number of unique values for both the object columns:\n",
    "The STABBR column is a good candidate to convert to Categorical as less than one\n",
    "percent of its values are unique:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Különbség a unique és a distinct között: Előbbi olyan értékeket jelent, amelyek csak egyszer fordulnak elő,\n",
    "a distinct pedig egy-egy érték előfordulási gyakoriságát jelenti."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "INSTNM    7535\n",
       "STABBR      59\n",
       "dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col2.select_dtypes(include=['object']).nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RELAFFIL        int8\n",
       "SATMTMID     float64\n",
       "CURROPER       int64\n",
       "INSTNM        object\n",
       "STABBR      category\n",
       "dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col2['STABBR'] = col2['STABBR'].astype('category')\n",
    "col2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index           80\n",
       "RELAFFIL      7535\n",
       "SATMTMID     60280\n",
       "CURROPER     60280\n",
       "INSTNM      660699\n",
       "STABBR       13576\n",
       "dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_mem = col2.memory_usage(deep=True)\n",
    "new_mem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's compare the original memory usage with our updated memory\n",
    "usage. The RELAFFIL column is, as expected, an eighth of its original, while the\n",
    "STABBR column has shrunk to just three percent of its original size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'62.41%'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#new_mem / original_mem\n",
    "str(round(new_mem.sum() *100/ original_mem.sum(),2))+'%' "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas defaults integer and float data types to 64 bits regardless of the maximum\n",
    "necessary size for the particular DataFrame. Integers, floats, and even booleans may be\n",
    "coerced to a different data type with the astype method and passing it the exact type,\n",
    "either as a string or specific object.\n",
    "Object data types can have a mix of strings, numerics, datetimes, or even other\n",
    "Python objects such as lists or tuples. For this reason, the object data type is sometimes\n",
    "referred to as a catch-all for a column of data that doesn't match any of the other data types.\n",
    "The vast majority of the time, though, object data type columns will all be strings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, the memory of each individual value in an object data type column is\n",
    "inconsistent. There is no predefined amount of memory for each value like the other data\n",
    "types. For pandas to extract the exact amount of memory of an object data type column, the\n",
    "deep parameter must be set to True in the memory_usage method.\n",
    "Object columns are targets for the largest memory savings. Pandas has an additional\n",
    "categorical data type that is not available in NumPy. When converting to category, pandas\n",
    "internally creates a mapping from integers to each unique string value. Thus, each string\n",
    "only needs to be kept a single time in memory. As you can see, this simple change of data\n",
    "type reduced memory usage by 97%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python 3 uses Unicode, a standardized character representation intended\n",
    "to encode all the world's writing systems.\n",
    "Not all columns can be coerced to the desired type. Take a look at the MENONLY column,\n",
    "which from the data dictionary appears to contain only 0/1 values. The actual data type of\n",
    "this column upon import unexpectedly turns out to be float64. The reason for this is that\n",
    "there happen to be missing values, denoted by np.nan. There is no integer representation\n",
    "for missing values. Any numeric column with even a single missing value must be a float.\n",
    "Furthermore, any column of an integer data type will automatically be coerced to a float if\n",
    "one of the values becomes missing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "college['MENONLY'].dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "#college['MENONLY'].astype(np.int8)\n",
    "#Cannot convert non-finite values (NA or inf) to integer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>HBCU</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.014238</td>\n",
       "      <td>0.118478</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.009213</td>\n",
       "      <td>0.095546</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WOMENONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005304</td>\n",
       "      <td>0.072642</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RELAFFIL</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.190975</td>\n",
       "      <td>0.393096</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATVRMID</th>\n",
       "      <td>1185.0</td>\n",
       "      <td>522.819409</td>\n",
       "      <td>68.578862</td>\n",
       "      <td>290.0</td>\n",
       "      <td>475.000000</td>\n",
       "      <td>510.00000</td>\n",
       "      <td>555.000000</td>\n",
       "      <td>765.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SATMTMID</th>\n",
       "      <td>1196.0</td>\n",
       "      <td>530.765050</td>\n",
       "      <td>73.469767</td>\n",
       "      <td>310.0</td>\n",
       "      <td>482.000000</td>\n",
       "      <td>520.00000</td>\n",
       "      <td>565.000000</td>\n",
       "      <td>785.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DISTANCEONLY</th>\n",
       "      <td>7164.0</td>\n",
       "      <td>0.005583</td>\n",
       "      <td>0.074519</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>2356.837940</td>\n",
       "      <td>5474.275871</td>\n",
       "      <td>0.0</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>412.50000</td>\n",
       "      <td>1929.500000</td>\n",
       "      <td>151558.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_WHITE</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.510207</td>\n",
       "      <td>0.286958</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.267500</td>\n",
       "      <td>0.55570</td>\n",
       "      <td>0.747875</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_BLACK</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.189997</td>\n",
       "      <td>0.224587</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.036125</td>\n",
       "      <td>0.10005</td>\n",
       "      <td>0.257700</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_HISP</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.161635</td>\n",
       "      <td>0.221854</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.027600</td>\n",
       "      <td>0.07140</td>\n",
       "      <td>0.198875</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_ASIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.033544</td>\n",
       "      <td>0.073777</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.01290</td>\n",
       "      <td>0.032700</td>\n",
       "      <td>0.9727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_AIAN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.013813</td>\n",
       "      <td>0.070196</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00260</td>\n",
       "      <td>0.007300</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NHPI</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.004569</td>\n",
       "      <td>0.033125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.9983</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_2MOR</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.023950</td>\n",
       "      <td>0.031288</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01750</td>\n",
       "      <td>0.033900</td>\n",
       "      <td>0.5333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_NRA</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.016086</td>\n",
       "      <td>0.050172</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.011700</td>\n",
       "      <td>0.9286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UGDS_UNKN</th>\n",
       "      <td>6874.0</td>\n",
       "      <td>0.045181</td>\n",
       "      <td>0.093440</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.01430</td>\n",
       "      <td>0.045400</td>\n",
       "      <td>0.9027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PPTUG_EF</th>\n",
       "      <td>6853.0</td>\n",
       "      <td>0.226639</td>\n",
       "      <td>0.246470</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.15040</td>\n",
       "      <td>0.376900</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CURROPER</th>\n",
       "      <td>7535.0</td>\n",
       "      <td>0.923291</td>\n",
       "      <td>0.266146</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTPELL</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.530643</td>\n",
       "      <td>0.225544</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.357800</td>\n",
       "      <td>0.52150</td>\n",
       "      <td>0.712900</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PCTFLOAN</th>\n",
       "      <td>6849.0</td>\n",
       "      <td>0.522211</td>\n",
       "      <td>0.283616</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.332900</td>\n",
       "      <td>0.58330</td>\n",
       "      <td>0.745000</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UG25ABV</th>\n",
       "      <td>6718.0</td>\n",
       "      <td>0.410021</td>\n",
       "      <td>0.228939</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.241500</td>\n",
       "      <td>0.40075</td>\n",
       "      <td>0.572275</td>\n",
       "      <td>1.0000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               count         mean          std    min         25%        50%  \\\n",
       "HBCU          7164.0     0.014238     0.118478    0.0    0.000000    0.00000   \n",
       "MENONLY       7164.0     0.009213     0.095546    0.0    0.000000    0.00000   \n",
       "WOMENONLY     7164.0     0.005304     0.072642    0.0    0.000000    0.00000   \n",
       "RELAFFIL      7535.0     0.190975     0.393096    0.0    0.000000    0.00000   \n",
       "SATVRMID      1185.0   522.819409    68.578862  290.0  475.000000  510.00000   \n",
       "SATMTMID      1196.0   530.765050    73.469767  310.0  482.000000  520.00000   \n",
       "DISTANCEONLY  7164.0     0.005583     0.074519    0.0    0.000000    0.00000   \n",
       "UGDS          6874.0  2356.837940  5474.275871    0.0  117.000000  412.50000   \n",
       "UGDS_WHITE    6874.0     0.510207     0.286958    0.0    0.267500    0.55570   \n",
       "UGDS_BLACK    6874.0     0.189997     0.224587    0.0    0.036125    0.10005   \n",
       "UGDS_HISP     6874.0     0.161635     0.221854    0.0    0.027600    0.07140   \n",
       "UGDS_ASIAN    6874.0     0.033544     0.073777    0.0    0.002500    0.01290   \n",
       "UGDS_AIAN     6874.0     0.013813     0.070196    0.0    0.000000    0.00260   \n",
       "UGDS_NHPI     6874.0     0.004569     0.033125    0.0    0.000000    0.00000   \n",
       "UGDS_2MOR     6874.0     0.023950     0.031288    0.0    0.000000    0.01750   \n",
       "UGDS_NRA      6874.0     0.016086     0.050172    0.0    0.000000    0.00000   \n",
       "UGDS_UNKN     6874.0     0.045181     0.093440    0.0    0.000000    0.01430   \n",
       "PPTUG_EF      6853.0     0.226639     0.246470    0.0    0.000000    0.15040   \n",
       "CURROPER      7535.0     0.923291     0.266146    0.0    1.000000    1.00000   \n",
       "PCTPELL       6849.0     0.530643     0.225544    0.0    0.357800    0.52150   \n",
       "PCTFLOAN      6849.0     0.522211     0.283616    0.0    0.332900    0.58330   \n",
       "UG25ABV       6718.0     0.410021     0.228939    0.0    0.241500    0.40075   \n",
       "\n",
       "                      75%          max  \n",
       "HBCU             0.000000       1.0000  \n",
       "MENONLY          0.000000       1.0000  \n",
       "WOMENONLY        0.000000       1.0000  \n",
       "RELAFFIL         0.000000       1.0000  \n",
       "SATVRMID       555.000000     765.0000  \n",
       "SATMTMID       565.000000     785.0000  \n",
       "DISTANCEONLY     0.000000       1.0000  \n",
       "UGDS          1929.500000  151558.0000  \n",
       "UGDS_WHITE       0.747875       1.0000  \n",
       "UGDS_BLACK       0.257700       1.0000  \n",
       "UGDS_HISP        0.198875       1.0000  \n",
       "UGDS_ASIAN       0.032700       0.9727  \n",
       "UGDS_AIAN        0.007300       1.0000  \n",
       "UGDS_NHPI        0.002500       0.9983  \n",
       "UGDS_2MOR        0.033900       0.5333  \n",
       "UGDS_NRA         0.011700       0.9286  \n",
       "UGDS_UNKN        0.045400       0.9027  \n",
       "PPTUG_EF         0.376900       1.0000  \n",
       "CURROPER         1.000000       1.0000  \n",
       "PCTPELL          0.712900       1.0000  \n",
       "PCTFLOAN         0.745000       1.0000  \n",
       "UG25ABV          0.572275       1.0000  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Additionally, it is possible to substitute string names in place of Python objects when referring to data types.\n",
    "college.describe(include=['int64', 'float64']).T\n",
    "college.describe(include=[np.int64, np.float64]).T\n",
    "college.describe(include=['int', 'float']).T\n",
    "college.describe(include=['number']).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60280"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'''Lastly, it is possible to see the enormous memory difference between the minimal\n",
    "RangeIndex and Int64Index, which stores every row index in memory:'''\n",
    "college.index = pd.Int64Index(college.index)\n",
    "college.index.memory_usage() # previously was just 80"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}