Loan Data - Training and Test Data Sets
For building the model, we will divide our data into two different data sets, namely training and testing datasets. The model will be built using the training set and then we will test it on the testing set to evaluate how our model is performing.
There are many ways in which we can split the data. If we had multi-year data, we could have used data for some years as training data and other years as testing data. Our data is for the same period (2016 Q1). We will use a simple approach to randomly divide the dataset into training and test set.
We can use the
"sample" command to randomly select certain index number and then use the selected index numbers to divide the dataset into training and testing dataset. Below is the code for doing this. In the code below we use 30% of the data for testing and rest of the 70% for training.
# Sample Indexes > indexes = sample(1:nrow(loandata), size=0.3*nrow(loandata)) > # Split data > data_test = loandata[indexes,] > dim(data_test)  17960 145 > data_train = loandata[-indexes,] > dim(data_train)  41909 145 >
We can now remove the original
loandata dataset from R to free up memory.
While building the model, we will emphasize that we avoid picking loans that can default as we don’t want to spoil our ROI. At the same time, we don’t want to pick up just a number of loans as we want to make a sizeable investment.
In the next few lessons we will focus on cleaning the dataset and then training the model.
- Credit Risk Modelling - Case Studies
- Classification vs. Regression Models
- Case Study - German Credit - Steps to Build a Predictive Model
- Import Credit Data Set in R
- German Credit Data : Data Preprocessing and Feature Selection in R
- Credit Modelling: Training and Test Data Sets
- Build the Predictive Model
- Logistic Regression Model in R
- Measure Model Performance in R Using ROCR Package
- Create a Confusion Matrix in R
- Credit Risk Modelling - Case Study- Lending Club Data
- Explore Loan Data in R - Loan Grade and Interest Rate
- Credit Risk Modelling - Required R Packages
- Loan Data - Training and Test Data Sets
- Data Cleaning in R - Part 1
- Data Cleaning in R - Part 2
- Data Cleaning in R - Part 3
- Data Cleaning in R - Part 5
- Remove Dimensions By Fitting Logistic Regression
- Create a Function and Prepare Test Data in R
- Building Credit Risk Model
- Credit Risk - Logistic Regression Model in R
- Support Vector Machine (SVM) Model in R
- Random Forest Model in R
- Extreme Gradient Boosting in R
- Predictive Modelling: Averaging Results from Multiple Models
- Predictive Modelling: Comparing Model Results
- How Insurance Companies Calculate Risk