pandas資料淨化實現刪除的專案實踐

2022-06-17 14:03:39

準備工作（匯入庫、匯入資料）

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import seaborn as  sns
sns.set_style("darkgrid")

list_csv = ['Amazon_top_selling_book.csv','breast_cancer_wisconsin.csv','diamonds.csv','insurance.csv','netflix_titles.csv','penguins.csv',
           'titanic.csv','winequality-red.csv']
dic_path = r'C:UserspandasDesktoptask228datasetsdatasets'
part_data = pd.read_csv(dic_path+'\'+list_csv[4])
part_data

	show_id	type	title	director	cast	country	date_added	release_year	rating	duration	listed_in	description
0	s1	Movie	Dick Johnson Is Dead	Kirsten Johnson	NaN	United States	September 25, 2021	2020	PG-13	90 min	Documentaries	As her father nears the end of his life, filmm...
1	s2	TV Show	Blood & Water	NaN	Ama Qamata, Khosi Ngema, Gail Mabalane, Thaban...	South Africa	September 24, 2021	2021	TV-MA	2 Seasons	International TV Shows, TV Dramas, TV Mysteries	After crossing paths at a party, a Cape Town t...
2	s3	TV Show	Ganglands	Julien Leclercq	Sami Bouajila, Tracy Gotoas, Samuel Jouy, Nabi...	NaN	September 24, 2021	2021	TV-MA	1 Season	Crime TV Shows, International TV Shows, TV Act...	To protect his family from a powerful drug lor...
3	s4	TV Show	Jailbirds New Orleans	NaN	NaN	NaN	September 24, 2021	2021	TV-MA	1 Season	Docuseries, Reality TV	Feuds, flirtations and toilet talk go down amo...
4	s5	TV Show	Kota Factory	NaN	Mayur More, Jitendra Kumar, Ranjan Raj, Alam K...	India	September 24, 2021	2021	TV-MA	2 Seasons	International TV Shows, Romantic TV Shows, TV ...	In a city of coaching centers known to train I...
...	...	...	...	...	...	...	...	...	...	...	...	...

8807 rows × 12 columns

檢測資料情況

Hint:該函數用於檢測任意DataFrame中缺失值情況

def missing_values_table(df):
        mis_val = df.isnull().sum()
        mis_val_percent = 100 * df.isnull().sum() / len(df)
        mis_val_table = pd.concat([mis_val, mis_val_percent], axis=1)
        mis_val_table_ren_columns = mis_val_table.rename(
        columns = {0 : 'Missing Values', 1 : '% of Total Values'})
        mis_val_table_ren_columns = mis_val_table_ren_columns[
            mis_val_table_ren_columns.iloc[:,1] != 0].sort_values(
        '% of Total Values', ascending=False).round(1)
        print ("Your selected dataframe has " + str(df.shape[1]) + " columns.n"      
            "There are " + str(mis_val_table_ren_columns.shape[0]) +
              " columns that have missing values.")
        return mis_val_table_ren_columns

missing_values_table(part_data)

Your selected dataframe has 12 columns.
There are 6 columns that have missing values.

	Missing Values	% of Total Values
director	2634	29.9
country	831	9.4
cast	825	9.4
date_added	10	0.1
rating	4	0.0
duration	3	0.0

DataFrame.drop(labels=None,axis=0, index=None, columns=None, inplace=False)

引數說明：

labels 就是要刪除的行列的名字，用列表給定
axis 預設為0，指刪除行，因此刪除columns時要指定axis=1；
index 直接指定要刪除的行
columns 直接指定要刪除的列
inplace=False，預設該刪除操作不改變原資料，而是返回一個執行刪除操作後的新dataframe；
inplace=True，則會直接在原資料上進行刪除操作，刪除後無法返回。

方式一：刪除指定行或列

labels+axis

demo = part_data.drop(['director'], axis=1)
missing_values_table(demo)

Your selected dataframe has 11 columns.
There are 5 columns that have missing values.

	Missing Values	% of Total Values
country	831	9.4
cast	825	9.4
date_added	10	0.1
rating	4	0.0
duration	3	0.0

方式二：利用boolean刪除滿足條件元素所在的行

df = df.drop(df[].index)

# 刪除release_year年份在2009年之前的行
demo = part_data.drop(part_data[part_data["release_year"]<2009].index)
demo.shape

(7624, 12)

到此這篇關於pandas資料淨化實現刪除的專案實踐的文章就介紹到這了,更多相關pandas資料淨化刪除內容請搜尋it145.com以前的文章或繼續瀏覽下面的相關文章希望大家以後多多支援it145.com！