Encoding Categorical Data in Python pandas

We have three columns with categorical data: LoanStatus, LoanAmountCategory, and CustomerLoyalty. To demonstrate encoding, we will apply it to the LoanStatus column. Since the values in LoanStatus are nominal without any intrinsic order, one-hot encoding is the appropriate technique. It avoids any ordinal implications that label encoding might introduce.

Before we do this, let’s check the various values in this column to ensure that there are no discrepancies. The following code gets us the unique values in LoanStatus column.

# print unique values in LoanStatus column

# ['pending', 'rejected', 'completed', 'approved', 'overdue', 'apprved', 'rejcted']
Categories (7, object): ['approved', 'apprved', 'completed', 'overdue', 'pending', 'rejcted', 'rejected']

We have a problem. Instead of the expected 5 categories, we have 7. This is because we have spelling mistakes in our data. We need to fix this.

To fix the spelling mistakes in the 'LoanStatus' column of our loan_data_copy DataFrame, we can use the .replace() method in Pandas. We want to correct 'apprved' to 'approved' and 'rejcted' to 'rejected'.

# Correcting the spelling mistakes
loan_data_cleaned['LoanStatus'] = loan_data_cleaned['LoanStatus'].replace({'apprved': 'approved', 'rejcted': 'rejected'})

# Verify the changes by printing unique values again

# ['pending', 'rejected', 'completed', 'approved', 'overdue']
Categories (5, object): ['approved', 'completed', 'overdue', 'pending', 'rejected']

Now, we can perform one-hot encoding on the LoanStatus column using pandas:

# Perform one-hot encoding on 'LoanStatus'
loan_status_encoded = pd.get_dummies(loan_data_cleaned['LoanStatus'], prefix='Status')

# Join the encoded DataFrame with the original one, dropping the original 'LoanStatus' column
loan_data_cleaned = loan_data_cleaned.join(loan_status_encoded)

# Optionally, you can drop the original 'LoanStatus' column if you no longer need it
loan_data_cleaned.drop('LoanStatus', axis=1, inplace=True)

# Verify the changes

Related Downloads

Learn the skills required to excel in data science and data analytics covering R, Python, machine learning, and AI.

Free Guides - Getting Started with R and Python

Enter your name and email address below and we will email you the guides for R programming and Python.

Saylient AI Logo

Take the Next Step in Your Data Career

Join our membership for lifetime unlimited access to all our data analytics and data science learning content and resources.