Credit Risk Evaluator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Credit Risk Evaluator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve the Data\n",
    "\n",
    "The data is located in the Challenge Files Folder:\n",
    "\n",
    "* `lending_data.csv`\n",
    "\n",
    "Import the data using Pandas. Display the resulting dataframe to confirm the import was successful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   loan_size  interest_rate  borrower_income  debt_to_income  num_of_accounts  \\\n",
       "0    10700.0          7.672            52800        0.431818                5   \n",
       "1     8400.0          6.692            43600        0.311927                3   \n",
       "2     9000.0          6.963            46100        0.349241                3   \n",
       "3    10700.0          7.664            52700        0.430740                5   \n",
       "4    10800.0          7.698            53000        0.433962                5   \n",
       "\n",
       "   derogatory_marks  total_debt  loan_status  \n",
       "0                 1       22800            0  \n",
       "1                 0       13600            0  \n",
       "2                 0       16100            0  \n",
       "3                 1       22700            0  \n",
       "4                 1       23000            0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the data\n",
    "lending_df = pd.read_csv(\"Resources/lending_data.csv\")\n",
    "lending_df.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 77536 entries, 0 to 77535\n",
      "Data columns (total 8 columns):\n",
      " #   Column            Non-Null Count  Dtype  \n",
      "---  ------            --------------  -----  \n",
      " 0   loan_size         77536 non-null  float64\n",
      " 1   interest_rate     77536 non-null  float64\n",
      " 2   borrower_income   77536 non-null  int64  \n",
      " 3   debt_to_income    77536 non-null  float64\n",
      " 4   num_of_accounts   77536 non-null  int64  \n",
      " 5   derogatory_marks  77536 non-null  int64  \n",
      " 6   total_debt        77536 non-null  int64  \n",
      " 7   loan_status       77536 non-null  int64  \n",
      "dtypes: float64(3), int64(5)\n",
      "memory usage: 4.7 MB\n"
     ]
    }
   ],
   "source": [
    "# checking the type of the value into the dataset\n",
    "lending_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict Model Performance\n",
    "\n",
    "You will be creating and comparing two models on this data: a Logistic Regression, and a Random Forests Classifier. Before you create, fit, and score the models, make a prediction as to which model you think will perform better. You do not need to be correct! \n",
    "\n",
    "Write down your prediction in the designated cells in your Jupyter Notebook, and provide justification for your educated guess."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Replace the text in this markdown cell with your predictions, and be sure to provide justification for your guess.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Training and Testing Sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   loan_size  interest_rate  borrower_income  debt_to_income  num_of_accounts  \\\n",
       "0    10700.0          7.672            52800        0.431818                5   \n",
       "1     8400.0          6.692            43600        0.311927                3   \n",
       "2     9000.0          6.963            46100        0.349241                3   \n",
       "3    10700.0          7.664            52700        0.430740                5   \n",
       "4    10800.0          7.698            53000        0.433962                5   \n",
       "\n",
       "   derogatory_marks  total_debt  \n",
       "0                 1       22800  \n",
       "1                 0       13600  \n",
       "2                 0       16100  \n",
       "3                 1       22700  \n",
       "4                 1       23000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Split the data into X_train, X_test, y_train, y_test\n",
    "from sklearn.model_selection import train_test_split\n",
    "y = lending_df['loan_status'].values\n",
    "X = lending_df.drop('loan_status', axis=1)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1], dtype=int64)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#scaling data based on the values\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "X_train_scaled = scaler.transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create, Fit and Compare Models\n",
    "\n",
    "Create a Logistic Regression model, fit it to the data, and print the model's score. Do the same for a Random Forest Classifier. You may choose any starting hyperparameters you like. \n",
    "\n",
    "Which model performed better? How does that compare to your prediction? Write down your results and thoughts in the designated markdown cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(max_iter=10000)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train a Logistic Regression model and print the model score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "classifier = LogisticRegression(max_iter=10000)\n",
    "classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(max_iter=10000)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Data Score: 0.9919177328380795\n",
      "Testing Data Score: 0.9924680148576145\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# Train a Logistic Regression model print the model score\n",
    "print(f\"Training Data Score: {classifier.score(X_train, y_train)}\")\n",
    "print(f\"Testing Data Score: {classifier.score(X_test, y_test)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[18699,    93],\n",
       "       [   53,   539]], dtype=int64)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_true = y_test\n",
    "y_pred = classifier.predict(X_test)\n",
    "confusion_matrix(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9694593479158069\n"
     ]
    }
   ],
   "source": [
    "# retrieving the accuracy\n",
    "tp,tn,fp,fn = confusion_matrix(y_true, y_pred).ravel()\n",
    "accuracy = (tp + tn) / (tp + tn + fp + fn) \n",
    "# (18663 + 563) / (18663 + 563 + 56 + 102)\n",
    "print(f\"Accuracy: {accuracy}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00     18792\n",
      "           1       0.85      0.91      0.88       592\n",
      "\n",
      "    accuracy                           0.99     19384\n",
      "   macro avg       0.93      0.95      0.94     19384\n",
      "weighted avg       0.99      0.99      0.99     19384\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_true, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a Random Forest Classifier model and print the model score\n",
    "from sklearn.ensemble import RandomForestClassifier\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "X_train_scaled = scaler.transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Score: 0.9975409272252029\n",
      "Testing Score: 0.9917457697069748\n"
     ]
    }
   ],
   "source": [
    "# Train a Random Forest Classifier model and print the model score\n",
    "clf = RandomForestClassifier(random_state=1, n_estimators=500).fit(X_train_scaled, y_train)\n",
    "print(f'Training Score: {clf.score(X_train_scaled, y_train)}')\n",
    "print(f'Testing Score: {clf.score(X_test_scaled, y_test)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.29066792e-01 2.74137485e-01 1.74521675e-01 1.58682971e-01\n",
      " 1.18902531e-01 8.06639163e-05 1.44607881e-01]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgLUlEQVR4nO3db2yV9f3/8dehpec4R8/Gv/4JpVSn0lqt5RThlBXjkMMqEsh0dHEUjGWmASel8Qa1ugExVvNVKCgt1qkdNyjVIIKxrhwzB3V0bnQ9zjizsShrg6epZaMFfrFIuX43Go87nhY5tfT6tDwfyZVwrn7O1fd1BcIzV0/PcViWZQkAAMBg4+weAAAA4JsQLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMF2v3AMPlwoUL+vTTTzVhwgQ5HA67xwEAAJfAsiydPn1aycnJGjdu8PsoYyZYPv30U6WkpNg9BgAAGIL29nZNmzZt0K+PmWCZMGGCpP4Tjo+Pt3kaAABwKXp6epSSkhL6f3wwYyZYvvwxUHx8PMECAMAo800v5+BFtwAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMF6s3QNgdJmx4U27Rxh2x59cbPcIAIBvwB0WAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYb0jBUlVVpbS0NLlcLnk8HjU1NQ269rXXXtPChQs1ZcoUxcfHy+v1qrGxMWxNbW2tHA5HxPb5558PZTwAADDGRB0s9fX1KikpUXl5uVpbW5WXl6f8/Hy1tbUNuP7w4cNauHChGhoa1NLSottvv11LlixRa2tr2Lr4+HgFg8GwzeVyDe2sAADAmBIb7RO2bNmioqIirV69WpJUWVmpxsZGVVdXq6KiImJ9ZWVl2OMnnnhC+/fv1xtvvKHs7OzQfofDocTExGjHAQAAV4Co7rCcO3dOLS0t8vl8Yft9Pp+OHDlySce4cOGCTp8+rYkTJ4btP3PmjFJTUzVt2jTdddddEXdgvq63t1c9PT1hGwAAGJuiCpauri719fUpISEhbH9CQoI6Ojou6RjPPPOMzp49q+XLl4f2zZw5U7W1tTpw4IDq6urkcrk0b948HTt2bNDjVFRUyO12h7aUlJRoTgUAAIwiQ3rRrcPhCHtsWVbEvoHU1dVp48aNqq+v19SpU0P7586dqxUrVigrK0t5eXl65ZVXdP311+vZZ58d9FhlZWXq7u4Obe3t7UM5FQAAMApE9RqWyZMnKyYmJuJuSmdnZ8Rdl6+rr69XUVGRXn31Vd1xxx0XXTtu3DjNnj37ondYnE6nnE7npQ8PAABGrajusMTFxcnj8cjv94ft9/v9ys3NHfR5dXV1uu+++7R7924tXrz4G7+PZVkKBAJKSkqKZjwAADBGRf1bQqWlpSosLFROTo68Xq9qamrU1tam4uJiSf0/qjlx4oR27dolqT9WVq5cqW3btmnu3LmhuzNXXXWV3G63JGnTpk2aO3eurrvuOvX09Gj79u0KBALasWPHcJ0nAAAYxaIOloKCAp08eVKbN29WMBhUZmamGhoalJqaKkkKBoNh78ny/PPP6/z581q7dq3Wrl0b2r9q1SrV1tZKkk6dOqUHHnhAHR0dcrvdys7O1uHDh3Xrrbd+y9MDAABjgcOyLMvuIYZDT0+P3G63uru7FR8fb/c4Y9aMDW/aPcKwO/7kN/+YEgBweVzq/998lhAAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMN6QgqWqqkppaWlyuVzyeDxqamoadO1rr72mhQsXasqUKYqPj5fX61VjY2PEur179yojI0NOp1MZGRnat2/fUEYDAABjUNTBUl9fr5KSEpWXl6u1tVV5eXnKz89XW1vbgOsPHz6shQsXqqGhQS0tLbr99tu1ZMkStba2htY0NzeroKBAhYWFev/991VYWKjly5frvffeG/qZAQCAMcNhWZYVzRPmzJmjWbNmqbq6OrQvPT1dy5YtU0VFxSUd48Ybb1RBQYF+9atfSZIKCgrU09Ojt956K7Tmxz/+sb7//e+rrq7uko7Z09Mjt9ut7u5uxcfHR3FGiMaMDW/aPcKwO/7kYrtHAIAr1qX+/x3VHZZz586ppaVFPp8vbL/P59ORI0cu6RgXLlzQ6dOnNXHixNC+5ubmiGMuWrToosfs7e1VT09P2AYAAMamqIKlq6tLfX19SkhICNufkJCgjo6OSzrGM888o7Nnz2r58uWhfR0dHVEfs6KiQm63O7SlpKREcSYAAGA0GdKLbh0OR9hjy7Ii9g2krq5OGzduVH19vaZOnfqtjllWVqbu7u7Q1t7eHsUZAACA0SQ2msWTJ09WTExMxJ2Pzs7OiDskX1dfX6+ioiK9+uqruuOOO8K+lpiYGPUxnU6nnE5nNOMDAIBRKqo7LHFxcfJ4PPL7/WH7/X6/cnNzB31eXV2d7rvvPu3evVuLF0e+wNHr9UYc8+DBgxc9JgAAuHJEdYdFkkpLS1VYWKicnBx5vV7V1NSora1NxcXFkvp/VHPixAnt2rVLUn+srFy5Utu2bdPcuXNDd1Kuuuoqud1uSdK6des0f/58PfXUU1q6dKn279+vt99+W+++++5wnScAABjFon4NS0FBgSorK7V582bdcsstOnz4sBoaGpSamipJCgaDYe/J8vzzz+v8+fNau3atkpKSQtu6detCa3Jzc7Vnzx69/PLLuvnmm1VbW6v6+nrNmTNnGE4RAACMdlG/D4upeB+WkcH7sAAAhtNleR8WAAAAOxAsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeLF2DwCMRjM2vGn3CMPu+JOL7R4BAAbFHRYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8WLtHgDA6DVjw5t2jzDsjj+52O4RAAyAOywAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHhDCpaqqiqlpaXJ5XLJ4/Goqalp0LXBYFD33nuvbrjhBo0bN04lJSURa2pra+VwOCK2zz//fCjjAQCAMSbqYKmvr1dJSYnKy8vV2tqqvLw85efnq62tbcD1vb29mjJlisrLy5WVlTXocePj4xUMBsM2l8sV7XgAAGAMijpYtmzZoqKiIq1evVrp6emqrKxUSkqKqqurB1w/Y8YMbdu2TStXrpTb7R70uA6HQ4mJiWEbAACAFGWwnDt3Ti0tLfL5fGH7fT6fjhw58q0GOXPmjFJTUzVt2jTdddddam1t/VbHAwAAY0dUwdLV1aW+vj4lJCSE7U9ISFBHR8eQh5g5c6Zqa2t14MAB1dXVyeVyad68eTp27Nigz+nt7VVPT0/YBgAAxqYhvejW4XCEPbYsK2JfNObOnasVK1YoKytLeXl5euWVV3T99dfr2WefHfQ5FRUVcrvdoS0lJWXI3x8AAJgtqmCZPHmyYmJiIu6mdHZ2Rtx1+VZDjRun2bNnX/QOS1lZmbq7u0Nbe3v7sH1/AABglqiCJS4uTh6PR36/P2y/3+9Xbm7usA1lWZYCgYCSkpIGXeN0OhUfHx+2AQCAsSnqT2suLS1VYWGhcnJy5PV6VVNTo7a2NhUXF0vqv/Nx4sQJ7dq1K/ScQCAgqf+FtZ999pkCgYDi4uKUkZEhSdq0aZPmzp2r6667Tj09Pdq+fbsCgYB27NgxDKcIAABGu6iDpaCgQCdPntTmzZsVDAaVmZmphoYGpaamSup/o7ivvydLdnZ26M8tLS3avXu3UlNTdfz4cUnSqVOn9MADD6ijo0Nut1vZ2dk6fPiwbr311m9xagAAYKyIOlgkac2aNVqzZs2AX6utrY3YZ1nWRY+3detWbd26dSijAACAKwCfJQQAAIw3pDssAABgcDM2vGn3CMPu+JOLbf3+3GEBAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYLxYuwcYDWZseNPuEYbd8ScX2z0CAACXjDssAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4/HhhwAwDPiQVODy4g4LAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4xEsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMNKViqqqqUlpYml8slj8ejpqamQdcGg0Hde++9uuGGGzRu3DiVlJQMuG7v3r3KyMiQ0+lURkaG9u3bN5TRAADAGBR1sNTX16ukpETl5eVqbW1VXl6e8vPz1dbWNuD63t5eTZkyReXl5crKyhpwTXNzswoKClRYWKj3339fhYWFWr58ud57771oxwMAAGNQ1MGyZcsWFRUVafXq1UpPT1dlZaVSUlJUXV094PoZM2Zo27ZtWrlypdxu94BrKisrtXDhQpWVlWnmzJkqKyvTggULVFlZGe14AABgDIoqWM6dO6eWlhb5fL6w/T6fT0eOHBnyEM3NzRHHXLRo0UWP2dvbq56enrANAACMTVEFS1dXl/r6+pSQkBC2PyEhQR0dHUMeoqOjI+pjVlRUyO12h7aUlJQhf38AAGC2Ib3o1uFwhD22LCti3+U+ZllZmbq7u0Nbe3v7t/r+AADAXLHRLJ48ebJiYmIi7nx0dnZG3CGJRmJiYtTHdDqdcjqdQ/6eAABg9IjqDktcXJw8Ho/8fn/Yfr/fr9zc3CEP4fV6I4558ODBb3VMAAAwdkR1h0WSSktLVVhYqJycHHm9XtXU1KitrU3FxcWS+n9Uc+LECe3atSv0nEAgIEk6c+aMPvvsMwUCAcXFxSkjI0OStG7dOs2fP19PPfWUli5dqv379+vtt9/Wu+++OwynCAAARruog6WgoEAnT57U5s2bFQwGlZmZqYaGBqWmpkrqf6O4r78nS3Z2dujPLS0t2r17t1JTU3X8+HFJUm5urvbs2aNHH31Ujz32mK699lrV19drzpw53+LUAADAWBF1sEjSmjVrtGbNmgG/VltbG7HPsqxvPOY999yje+65ZyjjAACAMY7PEgIAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxhtSsFRVVSktLU0ul0sej0dNTU0XXX/o0CF5PB65XC5dc8012rlzZ9jXa2tr5XA4IrbPP/98KOMBAIAxJupgqa+vV0lJicrLy9Xa2qq8vDzl5+erra1twPWffPKJ7rzzTuXl5am1tVWPPPKIHnroIe3duzdsXXx8vILBYNjmcrmGdlYAAGBMiY32CVu2bFFRUZFWr14tSaqsrFRjY6Oqq6tVUVERsX7nzp2aPn26KisrJUnp6ek6evSonn76ad19992hdQ6HQ4mJiUM8DQAAMJZFdYfl3Llzamlpkc/nC9vv8/l05MiRAZ/T3NwcsX7RokU6evSovvjii9C+M2fOKDU1VdOmTdNdd92l1tbWi87S29urnp6esA0AAIxNUQVLV1eX+vr6lJCQELY/ISFBHR0dAz6no6NjwPXnz59XV1eXJGnmzJmqra3VgQMHVFdXJ5fLpXnz5unYsWODzlJRUSG32x3aUlJSojkVAAAwigzpRbcOhyPssWVZEfu+af3/7p87d65WrFihrKws5eXl6ZVXXtH111+vZ599dtBjlpWVqbu7O7S1t7cP5VQAAMAoENVrWCZPnqyYmJiIuymdnZ0Rd1G+lJiYOOD62NhYTZo0acDnjBs3TrNnz77oHRan0ymn0xnN+AAAYJSK6g5LXFycPB6P/H5/2H6/36/c3NwBn+P1eiPWHzx4UDk5ORo/fvyAz7EsS4FAQElJSdGMBwAAxqiofyRUWlqq3/zmN3rppZf00Ucfaf369Wpra1NxcbGk/h/VrFy5MrS+uLhY//73v1VaWqqPPvpIL730kl588UU9/PDDoTWbNm1SY2OjPv74YwUCARUVFSkQCISOCQAArmxR/1pzQUGBTp48qc2bNysYDCozM1MNDQ1KTU2VJAWDwbD3ZElLS1NDQ4PWr1+vHTt2KDk5Wdu3bw/7leZTp07pgQceUEdHh9xut7Kzs3X48GHdeuutw3CKAABgtIs6WCRpzZo1WrNmzYBfq62tjdh322236a9//eugx9u6dau2bt06lFEAAMAVgM8SAgAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGI1gAAIDxCBYAAGA8ggUAABiPYAEAAMYjWAAAgPEIFgAAYDyCBQAAGI9gAQAAxiNYAACA8QgWAABgPIIFAAAYj2ABAADGG1KwVFVVKS0tTS6XSx6PR01NTRddf+jQIXk8HrlcLl1zzTXauXNnxJq9e/cqIyNDTqdTGRkZ2rdv31BGAwAAY1DUwVJfX6+SkhKVl5ertbVVeXl5ys/PV1tb24DrP/nkE915553Ky8tTa2urHnnkET300EPau3dvaE1zc7MKCgpUWFio999/X4WFhVq+fLnee++9oZ8ZAAAYM6IOli1btqioqEirV69Wenq6KisrlZKSourq6gHX79y5U9OnT1dlZaXS09O1evVq3X///Xr66adDayorK7Vw4UKVlZVp5syZKisr04IFC1RZWTnkEwMAAGNHbDSLz507p5aWFm3YsCFsv8/n05EjRwZ8TnNzs3w+X9i+RYsW6cUXX9QXX3yh8ePHq7m5WevXr49Yc7Fg6e3tVW9vb+hxd3e3JKmnpyeaU7okF3r/37Af025DvU5ci35ch35ch69wLfC/+PsQ/XEty7rouqiCpaurS319fUpISAjbn5CQoI6OjgGf09HRMeD68+fPq6urS0lJSYOuGeyYklRRUaFNmzZF7E9JSbnU07miuSvtnsAcXIt+XId+XIevcC3wvy7334fTp0/L7XYP+vWoguVLDocj7LFlWRH7vmn91/dHe8yysjKVlpaGHl+4cEH/+c9/NGnSpIs+z2Q9PT1KSUlRe3u74uPj7R7HNlyHflyHr3At+nEd+nEdvjIWroVlWTp9+rSSk5Mvui6qYJk8ebJiYmIi7nx0dnZG3CH5UmJi4oDrY2NjNWnSpIuuGeyYkuR0OuV0OsP2fe9737vUUzFafHz8qP2LN5y4Dv24Dl/hWvTjOvTjOnxltF+Li91Z+VJUL7qNi4uTx+OR3+8P2+/3+5Wbmzvgc7xeb8T6gwcPKicnR+PHj7/omsGOCQAArixR/0iotLRUhYWFysnJkdfrVU1Njdra2lRcXCyp/0c1J06c0K5duyRJxcXFeu6551RaWqpf/OIXam5u1osvvqi6urrQMdetW6f58+frqaee0tKlS7V//369/fbbevfdd4fpNAEAwGgWdbAUFBTo5MmT2rx5s4LBoDIzM9XQ0KDU1FRJUjAYDHtPlrS0NDU0NGj9+vXasWOHkpOTtX37dt19992hNbm5udqzZ48effRRPfbYY7r22mtVX1+vOXPmDMMpjh5Op1O//vWvI37UdaXhOvTjOnyFa9GP69CP6/CVK+laOKxv+j0iAAAAm/FZQgAAwHgECwAAMB7BAgAAjEewAAAA4xEshqiqqlJaWppcLpc8Ho+amprsHmnEHT58WEuWLFFycrIcDodef/11u0eyRUVFhWbPnq0JEyZo6tSpWrZsmf7xj3/YPdaIq66u1s033xx6Qyyv16u33nrL7rFsV1FRIYfDoZKSErtHGXEbN26Uw+EI2xITE+0eyxYnTpzQihUrNGnSJH3nO9/RLbfcopaWFrvHuqwIFgPU19erpKRE5eXlam1tVV5envLz88N+PfxKcPbsWWVlZem5556zexRbHTp0SGvXrtWf/vQn+f1+nT9/Xj6fT2fPnrV7tBE1bdo0Pfnkkzp69KiOHj2qH/3oR1q6dKk+/PBDu0ezzV/+8hfV1NTo5ptvtnsU29x4440KBoOh7YMPPrB7pBH33//+V/PmzdP48eP11ltv6e9//7ueeeaZMfNu74Ph15oNMGfOHM2aNUvV1dWhfenp6Vq2bJkqKipsnMw+DodD+/bt07Jly+wexXafffaZpk6dqkOHDmn+/Pl2j2OriRMn6v/+7/9UVFRk9ygj7syZM5o1a5aqqqr0+OOP65ZbbrnoJ9qPRRs3btTrr7+uQCBg9yi22rBhg/74xz9ecXfiucNis3PnzqmlpUU+ny9sv8/n05EjR2yaCibp7u6W1P+f9ZWqr69Pe/bs0dmzZ+X1eu0exxZr167V4sWLdccdd9g9iq2OHTum5ORkpaWl6Wc/+5k+/vhju0cacQcOHFBOTo5++tOfaurUqcrOztYLL7xg91iXHcFis66uLvX19UV80GNCQkLEB0LiymNZlkpLS/XDH/5QmZmZdo8z4j744AN997vfldPpVHFxsfbt26eMjAy7xxpxe/bsUUtLyxV7x/VLc+bM0a5du9TY2KgXXnhBHR0dys3N1cmTJ+0ebUR9/PHHqq6u1nXXXafGxkYVFxfroYceCn0kzlgV9Vvz4/JwOBxhjy3LitiHK8+DDz6ov/3tb1fs52rdcMMNCgQCOnXqlPbu3atVq1bp0KFDV1S0tLe3a926dTp48KBcLpfd49gqPz8/9OebbrpJXq9X1157rX7729+qtLTUxslG1oULF5STk6MnnnhCkpSdna0PP/xQ1dXVWrlypc3TXT7cYbHZ5MmTFRMTE3E3pbOzM+KuC64sv/zlL3XgwAG98847mjZtmt3j2CIuLk4/+MEPlJOTo4qKCmVlZWnbtm12jzWiWlpa1NnZKY/Ho9jYWMXGxurQoUPavn27YmNj1dfXZ/eItrn66qt100036dixY3aPMqKSkpIioj09PX3M/6IGwWKzuLg4eTwe+f3+sP1+v1+5ubk2TQU7WZalBx98UK+99pp+//vfKy0tze6RjGFZlnp7e+0eY0QtWLBAH3zwgQKBQGjLycnRz3/+cwUCAcXExNg9om16e3v10UcfKSkpye5RRtS8efMi3urgn//8Z+hDiMcqfiRkgNLSUhUWFionJ0der1c1NTVqa2tTcXGx3aONqDNnzuhf//pX6PEnn3yiQCCgiRMnavr06TZONrLWrl2r3bt3a//+/ZowYULo7pvb7dZVV11l83Qj55FHHlF+fr5SUlJ0+vRp7dmzR3/4wx/0u9/9zu7RRtSECRMiXr909dVXa9KkSVfc65oefvhhLVmyRNOnT1dnZ6cef/xx9fT0aNWqVXaPNqLWr1+v3NxcPfHEE1q+fLn+/Oc/q6amRjU1NXaPdnlZMMKOHTus1NRUKy4uzpo1a5Z16NAhu0cace+8844lKWJbtWqV3aONqIGugSTr5Zdftnu0EXX//feH/k1MmTLFWrBggXXw4EG7xzLCbbfdZq1bt87uMUZcQUGBlZSUZI0fP95KTk62fvKTn1gffvih3WPZ4o033rAyMzMtp9NpzZw506qpqbF7pMuO92EBAADG4zUsAADAeAQLAAAwHsECAACMR7AAAADjESwAAMB4BAsAADAewQIAAIxHsAAAAOMRLAAAwHgECwAAMB7BAgAAjEewAAAA4/1/dMTcKzYpcD0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "features = clf.feature_importances_\n",
    "print(features)\n",
    "plt.bar(x = range(len(features)), height=features)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True,  True,  True, False, False,  True])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_selection import SelectFromModel\n",
    "sel = SelectFromModel(clf)\n",
    "sel.fit(X_train_scaled, y_train)\n",
    "sel.get_support()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\madin\\anaconda3\\envs\\mlenv\\lib\\site-packages\\sklearn\\base.py:444: UserWarning: X has feature names, but SelectFromModel was fitted without feature names\n",
      "  f\"X has feature names, but {self.__class__.__name__} was fitted without\"\n"
     ]
    }
   ],
   "source": [
    "X_selected_train, X_selected_test, y_train, y_test = train_test_split(sel.transform(X), y, random_state=1)\n",
    "scaler = StandardScaler().fit(X_selected_train)\n",
    "X_selected_train_scaled = scaler.transform(X_selected_train)\n",
    "X_selected_test_scaled = scaler.transform(X_selected_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Score: 0.9942908240473243\n",
      "Testing Score: 0.9936545604622369\n"
     ]
    }
   ],
   "source": [
    "clf = LogisticRegression().fit(X_train_scaled, y_train)\n",
    "print(f'Training Score: {clf.score(X_train_scaled, y_train)}')\n",
    "print(f'Testing Score: {clf.score(X_test_scaled, y_test)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Score: 0.9942908240473243\n",
      "Testing Score: 0.9936545604622369\n"
     ]
    }
   ],
   "source": [
    "clf = LogisticRegression().fit(X_selected_train_scaled, y_train)\n",
    "print(f'Training Score: {clf.score(X_selected_train_scaled, y_train)}')\n",
    "print(f'Testing Score: {clf.score(X_selected_test_scaled, y_test)}')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analysis\n",
    "My prediction is wrong. As it can be seen that both models testing score have not significant different. The LRM has a testing score of 0.9918 while the RFM's testing score is 0.9917. However, if considering the training score, the one displayed with the RFM is higher than the one detected with the LRM. Overall, both models show positive trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Which model performed better? How does that compare to your prediction? Replace the text in this markdown cell with your answers to these questions.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mlenv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "299d3c6e69308ae7c0c2a9a0f9e33b7e211f098ed95d2a4a80a0f3d95ade8079"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}