Skip to content

Latest commit

 

History

History
158 lines (82 loc) · 3.92 KB

README.md

File metadata and controls

158 lines (82 loc) · 3.92 KB

Dealing with Categorical Variables - Lab



Introduction


In this lab, you'll explore the Ames Housing dataset and identify numeric and categorical variables. Then you'll transform some categorical data and use it in a multiple regression model.


Objectives


You will be able to:


    

  • Determine whether variables are categorical or numeric
    • 

  • Use one-hot encoding to create dummy variables
    • 



Step 1: Load the Ames Housing Dataset


Import pandas, and use it to load the file ames.csv into a dataframe called ames. If you pass in the argument index_col=0 this will set the "Id" feature as the index.


# Your code here - load the dataset


Visually inspect ames (it's ok if you can't see all of the columns).


# Your code here


Go ahead and drop all columns with missing data, to simplify the problem. Remember that you can use the dropna method (documentation here).


# Your code here - drop columns with missing data


Step 2: Identify Numeric and Categorical Variables


The file data_description.txt, located in this repository, has a full description of all variables.


Using this file as well as pandas techniques, identify the following predictors:


    

  1. A continuous numeric predictor
    2. 

  2. A discrete numeric predictor
    3. 

  3. A string categorical predictor
    4. 

  4. A discrete categorical predictor
    5. 



(Note that SalePrice is the target variable and should not be selected as a predictor.)


For each of these predictors, visualize the relationship between the predictor and SalePrice using an appropriate plot.


Finding these will take some digging -- don't be discouraged if they're not immediately obvious. The Ames Housing dataset is a lot more complex than the Auto MPG dataset. There is also no single right answer here.


Continuous Numeric Predictor


# Your code here - continuous numeric predictor


Discrete Numeric Predictor


# Your code here - discrete numeric predictor


String Categorical Predictor


# Your code here - string categorical predictor


Discrete Categorical Predictor


# Your code here - discrete categorical predictor


Step 3: Build a Multiple Regression Model with Your Chosen Predictors


Choose the best-looking 3 out of 4 predictors to include in your model.


Make sure that you one-hot encode your categorical predictor(s) (regardless of whether the current data type is a string or number) first.


# Your code here - prepare X and y, including one-hot encoding


# Your answer here - which category or categories were dropped?


# Your code here - build a regression model and display results


Step 4: Create Partial Regression Plots for Features


For each feature of the regression above (including the dummy features), plot the partial regression.


# Your code here - create partial regression plots


Step 5: Calculate an Error-Based Metric


In addition to the adjusted R-Squared that we can see in the model summary, calculate either MAE or RMSE for this model.


# Your code here - calculate an error-based metric


Step 6: Summarize Findings


Between the model results, partial regression plots, and error-based metric, what does this model tell you? What would your next steps be to improve the model?


# Your answer here


Level Up (Optional)


Try transforming X using scikit-learn and fitting a scikit-learn linear regression as well. If there are any differences in the result, investigate them.


# Your code here


Summary


In this lab, you practiced your knowledge of categorical variables on the Ames Housing dataset! Specifically, you practiced distinguishing numeric and categorical data. You then created dummy variables using one hot encoding in order to build a multiple regression model.