Data integration and manipulation with cuDF and pandas: Worked Example — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)

{ "title": "NVIDIA-Certified Associate: Accelerated Data Science - Data Integration and Manipulation with cuDF and pandas: Worked Example", "category": "NVIDIA-Certified Associate: Accelerated Data Science", "hashtags": "NVIDIA, accelerated-data-science, cuDF, pandas, data-manipulation", "content": "

Data Integration and Manipulation with cuDF and pandas: Worked Example

In the NVIDIA-Certified Associate: Accelerated Data Science exam, data manipulation and preparation form a critical component. This example focuses on integrating and manipulating data using cuDF and pandas, two powerful libraries for GPU-accelerated and CPU-based data processing respectively.

Scenario

Suppose you are working with two datasets:

• Customer Info: Contains customer IDs, names, and demographic details.
• Transaction Records: Contains customer IDs, transaction dates, and amounts.

Your task is to merge these datasets, clean missing values, and prepare the data for further analysis — leveraging GPU acceleration with cuDF where possible.

Step 1: Import Libraries and Load Data

First, import cudf and pandas. Use pandas to load CSV files if data is on disk, then convert to cuDF DataFrames for GPU processing.

Code snippet

import cudf import pandas as pd

Load CSVs using pandas

customer_pdf = pd.read_csv('customer_info.csv') transaction_pdf = pd.read_csv('transaction_records.csv')

Convert pandas DataFrames to cuDF DataFrames for GPU acceleration

customer_cdf = cudf.from_pandas(customer_pdf) transaction_cdf = cudf.from_pandas(transaction_pdf)

Step 2: Data Inspection

Check for missing values and data types to understand cleaning needs.

Code snippet

Check for missing values

print(customer_cdf.isnull().sum()) print(transaction_cdf.isnull().sum())

Check data types

print(customer_cdf.dtypes) print(transaction_cdf.dtypes)

Step 3: Data Cleaning

Suppose the 'name' column in customer_cdf has some missing values. We can fill these with a placeholder or drop those rows depending on analysis needs.

Code snippet

Fill missing names with 'Unknown'

customer_cdf['name'] = customer_cdf['name'].fillna('Unknown')

Drop transactions with missing amounts

transaction_cdf = transaction_cdf.dropna(subset=['amount'])

Step 4: Data Integration (Join)

Perform an inner join on customer_id to combine customer demographics with their transactions.

Code snippet

merged_cdf = customer_cdf.merge(transaction_cdf, on='customer_id', how='inner')

Step 5: Feature Engineering

Create a new feature transaction_year extracted from the transaction_date for temporal analysis.

Code snippet

Convert transaction_date to datetime

merged_cdf['transaction_date'] = cudf.to_datetime(merged_cdf['transaction_date'])

Extract year

merged_cdf['transaction_year'] = merged_cdf['transaction_date'].dt.year

Step 6: Convert Back to pandas (if needed)

For compatibility with libraries that do not support cuDF, convert back to pandas DataFrame.

Code snippet

final_pdf = merged_cdf.to_pandas()

Summary

This step-by-step example demonstrates how to:

• Load data using pandas and convert to cuDF for GPU acceleration
• Inspect and clean missing data efficiently on GPU
• Integrate datasets via joins using cuDF
• Engineer features leveraging cuDF datetime operations
• Convert data back to pandas if necessary for downstream tasks

Mastering these operations is essential for the Data Manipulation and Preparation section of the NVIDIA-Certified Associate: Accelerated Data Science exam, enabling efficient handling of large datasets with GPU acceleration.

For further details on RAPIDS cuDF and pandas interoperability, visit the official RAPIDS documentation: https://rapids.ai/start.html

" }

More in this topic

Data integration and manipulation with cuDF and pandas: Quick Reference — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Feature engineering for numerical and categorical variables — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Dimensionality reduction and data sampling: Worked Example — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Efficient processing and storage with Parquet — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data integration and manipulation with cuDF and pandas: Practice Questions — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data cleaning, quality handling, and governance: Worked Example — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data cleaning, quality handling, and governance: Common Mistakes — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data cleaning, quality handling, and governance: Practice Questions — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data integration and manipulation with cuDF and pandas — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data integration and manipulation with cuDF and pandas: Common Mistakes — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Dimensionality reduction and data sampling: Practice Questions — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data cleaning, quality handling, and governance — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data cleaning, quality handling, and governance: Quick Reference — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)GPU-accelerated ETL with RAPIDS, Dask, or Spark — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Dimensionality reduction and data sampling — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Dimensionality reduction and data sampling: Quick Reference — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Dimensionality reduction and data sampling: Common Mistakes — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Handling class imbalance and generating synthetic data — Data Manipulation and Preparation (NVIDIA-Certified Associate: Accelerated Data Science)Data Manipulation and Preparation — NVIDIA-Certified Associate: Accelerated Data Science