Data cleansing and preprocessing with cuDF and pandas: Worked Example — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)

Data Cleansing and Preprocessing with cuDF and pandas: Worked Example

In the NVIDIA-Certified Professional: Accelerated Data Science exam, data preparation is a critical skill, particularly the ability to cleanse and preprocess data efficiently using GPU-accelerated libraries such as cuDF alongside traditional tools like pandas. This worked example demonstrates a step-by-step approach to cleaning and preprocessing a realistic dataset using both libraries, highlighting the reasoning and concrete steps involved.

Scenario

Suppose you have a dataset containing customer information for a retail company. The dataset includes columns such as CustomerID, Age, AnnualIncome, and PurchaseAmount. The data contains missing values, inconsistent formats, and outliers that must be addressed before analysis or modeling.

Step 1: Loading the Data

First, load the dataset using cuDF to leverage GPU acceleration for large data:

import cudf

df = cudf.read_csv('customer_data.csv')

For smaller datasets or initial exploration, pandas can be used:

import pandas as pd

df_pd = pd.read_csv('customer_data.csv')

Step 2: Inspecting the Data

Check for missing values and data types:

print(df.info()) print(df.isnull().sum())

This reveals which columns have missing entries and their data types.

Step 3: Handling Missing Values

For numerical columns like Age and AnnualIncome, fill missing values with the median to reduce bias from outliers:

median_age = df['Age'].median() df['Age'] = df['Age'].fillna(median_age)

Similarly, for AnnualIncome:

median_income = df['AnnualIncome'].median() df['AnnualIncome'] = df['AnnualIncome'].fillna(median_income)

For categorical or ID columns, you might drop rows or fill with a placeholder depending on context.

Step 4: Standardizing Formats

Ensure consistent data types and formats. For example, convert CustomerID to string if it contains leading zeros:

df['CustomerID'] = df['CustomerID'].astype('str')

For date columns, convert to datetime format (if applicable):

df['SignupDate'] = cudf.to_datetime(df['SignupDate'], errors='coerce')

Step 5: Detecting and Handling Outliers

Calculate the interquartile range (IQR) to identify outliers in PurchaseAmount:

Q1 = df['PurchaseAmount'].quantile(0.25) Q3 = df['PurchaseAmount'].quantile(0.75) IQR = Q3 - Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR

Filter out outliers

filtered_df = df[(df['PurchaseAmount'] >= lower_bound) & (df['PurchaseAmount'] <= upper_bound)]

Step 6: Feature Scaling (Optional)

To prepare for modeling, scale numerical features. Using cuDF, you can perform min-max scaling:

min_income = filtered_df['AnnualIncome'].min() max_income = filtered_df['AnnualIncome'].max() filtered_df['AnnualIncome_Scaled'] = (filtered_df['AnnualIncome'] - min_income) / (max_income - min_income)

Step 7: Converting Between cuDF and pandas

Sometimes, you may want to switch between cuDF and pandas depending on the operation or library compatibility:

# cuDF to pandas df_pd = filtered_df.to_pandas()

pandas to cuDF

df_cudf = cudf.DataFrame.from_pandas(df_pd)

Summary

This example illustrates how to use cuDF for GPU-accelerated data cleansing and preprocessing, including handling missing values, standardizing formats, detecting outliers, and scaling features. Integrating pandas allows flexibility for operations not yet supported in cuDF or for smaller datasets. Mastery of these steps is essential for efficient data preparation in accelerated data science workflows.

For more details on cuDF and RAPIDS libraries, visit the official documentation at https://rapids.ai/.

More in this topic

Data cleansing and preprocessing with cuDF and pandas: Practice Questions — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Data cleansing and preprocessing with cuDF and pandas: Common Mistakes — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Monitoring pipeline bottlenecks — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Data Preparation — NVIDIA-Certified Professional: Accelerated Data ScienceData cleansing and preprocessing with cuDF and pandas — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Data cleansing and preprocessing with cuDF and pandas: Quick Reference — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Generating synthetic data with cuDF and RAPIDS — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)Transforming and standardizing features — Data Preparation (NVIDIA-Certified Professional: Accelerated Data Science)