Kaggle泰坦尼克預測

基於之前寫的文章titanic初探，本文採用隨機森林演算法對生存率進行預測，最終kaggle提交結果準確率為0.78468，總體還說的過去。至此kaggle的練習項目Titanic已全部分析完畢，如果錯誤或遺漏請指正，不勝感激。

一、定義問題

本質上是對生存幾率進行預測，產生的結果無非兩種：生存或者死亡，所以是一個二分類問題，理論上可以採用分類演算法進行計算，本文計劃採用隨機森林，原因是兩個:

二、導入數據

#導入需要的包，演算法採用隨機森林import pandas as pdimport numpy as npfrom sklearn.ensemble import RandomForestClassifiertrain_data=pd.read_csv(rd:work rain.csv)test_data=pd.read_csv(rd:work est.csv)train_data.head()#查看訓練集文件頭部五行內容test_data.head()#查看測試集文件頭部五行內容，少一列Survived

train_data.info()#查看索引、數據類型和內存信息<class pandas.core.frame.DataFrame>RangeIndex: 891 entries, 0 to 890Data columns (total 12 columns):PassengerId 891 non-null int64Survived 891 non-null int64Pclass 891 non-null int64Name 891 non-null objectSex 891 non-null objectAge 714 non-null float64SibSp 891 non-null int64Parch 891 non-null int64Ticket 891 non-null objectFare 891 non-null float64Cabin 204 non-null objectEmbarked 889 non-null objectdtypes: float64(2), int64(5), object(5)memory usage: 83.6+ KBtest_data[Survived]=0#測試集加入Survived列，此時Survived列處於test.csv的最後列Survived=test_data.pop(Survived)#彈出test_data.insert(1,Survived,Survived)#調整到第二列，與train文件順序保持一致all_data=train_data.append(test_data,)#訓練集與測試集合併，後續進行相同的特徵工程處理all_data.info()#總覽數據，為特徵工程做準備<class pandas.core.frame.DataFrame>Int64Index: 1309 entries, 0 to 417Data columns (total 12 columns):PassengerId 1309 non-null int64Survived 1309 non-null int64Pclass 1309 non-null int64Name 1309 non-null objectSex 1309 non-null objectAge 1046 non-null float64SibSp 1309 non-null int64Parch 1309 non-null int64Ticket 1309 non-null objectFare 1308 non-null float64Cabin 295 non-null objectEmbarked 1307 non-null objectdtypes: float64(2), int64(5), object(5)memory usage: 132.9+ KBall_data.isnull().sum().sort_values(ascending=False)#查看缺失數據Cabin 1014Age 263Embarked 2Fare 1Ticket 0Parch 0SibSp 0Sex 0Name 0Pclass 0Survived 0PassengerId 0dtype: int64

三、特徵工程

（1）、缺失值處理

Cabin特徵缺失太多，可以利用有無記錄作為分類依據做二值化處理

all_data.loc[all_data[Cabin].isnull(),Cabin]=0all_data.loc[all_data[Cabin].notnull(),Cabin]=1#查看處理後的結果all_data.Cabin.isnull().sum()0

Fare特徵僅缺失一個，用平均數進行填充

all_data[Fare]=all_data[Fare].fillna(all_data[Fare].mean())all_data.Fare.isnull().sum()0all_data.Embarked.value_counts()S 914C 270Q 123Name: Embarked, dtype: int64

Embarked缺失兩個，用出現次數最多的S進行填充

all_data[Embarked]=all_data[Embarked].fillna(S)all_data.Embarked.isnull().sum()0

Age特徵經過之前文章的分析，非常重要，計劃在將其他定性特徵轉換為數值特徵之後利用隨機森林進行缺失值的預測填充

（2）、定性變數轉換

(all_data.dtypes==object).sort_values(ascending=False)#查看數據類型為對象的特徵，目前實例中的非數值特徵少，如果多的話，必須這麼查找

Embarked TrueTicket TrueSex TrueName TrueCabin FalseFare FalseParch FalseSibSp FalseAge FalsePclass FalseSurvived FalsePassengerId Falsedtype: bool

all_data.Embarked.value_counts()#查看Embarke的取值和類別

S 916C 270Q 123Name: Embarked, dtype: int64

all_data.Ticket.value_counts()#查看Ticket的取值和類別

CA. 2343 11CA 2144 81601 8S.O.C. 14879 7347082 73101295 7347077 7PC 17608 7382652 6347088 619950 6113781 64133 516966 5W./C. 6608 5PC 17757 5220845 5349909 5113503 517421 412749 4LINE 424160 4C.A. 2315 4C.A. 34651 436928 4W./C. 6607 4SC/Paris 2123 4113760 4C.A. 33112 4 ..330980 1STON/O 2. 3101289 17540 1236171 117764 1113801 1345501 1364506 1SO/C 14885 1349226 1324669 165304 1C 17368 112233 12621 1A/4. 34244 1244310 1364858 1368702 1113800 1350060 1STON/O 2. 3101285 1680 1A/4 45380 12669 1C 7077 1218629 1370377 1347086 1330923 1Name: Ticket, Length: 929, dtype: int64all_data.Sex.value_counts()#查看Sex的取值和類別

male 843female 466Name: Sex, dtype: int64

all_data.Name.value_counts()#查看Name的取值和類別

Kelly, Mr. James 2Connolly, Miss. Kate 2Dennis, Mr. Samuel 1Baxter, Mrs. James (Helene DeLaudeniere Chaput) 1Flegenheim, Mrs. Alfred (Antoinette) 1Franklin, Mr. Thomas Parham 1Berriman, Mr. William John 1Hendekovic, Mr. Ignjac 1McCoy, Miss. Alicia 1Morrow, Mr. Thomas Rowan 1Finoli, Mr. Luigi 1Ali, Mr. Ahmed 1Peltomaki, Mr. Nikolai Johannes 1Wheeler, Mr. Edwin Frederick"" 1Baccos, Mr. Raffull 1Albimona, Mr. Nassef Cassem 1Chaudanson, Miss. Victorine 1Clark, Mr. Walter Miller 1Silverthorne, Mr. Spencer Victor 1Birnbaum, Mr. Jakob 1Hays, Mr. Charles Melville 1Ilmakangas, Miss. Pieta Sofia 1Jefferys, Mr. Clifford Thomas 1Douglas, Mrs. Walter Donald (Mahala Dutton) 1Sage, Mr. John George 1Collyer, Miss. Marjorie "Lottie" 1Veal, Mr. James 1Mineff, Mr. Ivan 1Simonius-Blumer, Col. Oberst Alfons 1Osen, Mr. Olaf Elon 1 ..Minahan, Mrs. William Edward (Lillian E Thorpe) 1McDermott, Miss. Brigdet Delia 1Sage, Miss. Stella Anna 1Edvardsson, Mr. Gustaf Hjalmar 1Butler, Mr. Reginald Fenton 1Richard, Mr. Emile 1Laitinen, Miss. Kristina Sofia 1Mallet, Master. Andre 1Quick, Mrs. Frederick Charles (Jane Richards) 1Renouf, Mr. Peter Henry 1Graham, Mr. George Edward 1Colbert, Mr. Patrick 1Nicholson, Mr. Arthur Ernest 1Kent, Mr. Edward Austin 1Risien, Mrs. Samuel (Emma) 1Hegarty, Miss. Hanora "Nora" 1Earnshaw, Mrs. Boulton (Olive Potter) 1Byles, Rev. Thomas Roussel Davids 1Andersson, Miss. Ingeborg Constanzia 1Daniel, Mr. Robert Williams 1Reynaldo, Ms. Encarnacion 1Holverson, Mrs. Alexander Oskar (Mary Aline Towner) 1Lefebre, Miss. Jeannie 1Saade, Mr. Jean Nassr 1Van Impe, Miss. Catharina 1Daly, Mr. Eugene Patrick 1Allen, Miss. Elisabeth Walton 1Hays, Miss. Margaret Bechstein 1Riihivouri, Miss. Susanna Juhantytar Sanni"" 1Pallas y Castello, Mr. Emilio 1Name: Name, Length: 1307, dtype: int64

定性特徵查看結果表明Name和Ticket取值範圍太多，為簡化 難度，後續模型不予採納

Sex特徵利用啞變數進行定性轉換

all_data[Sex]=pd.get_dummies(all_data[Sex]).astype(int)#Age啞變數處理

all_data.Sex.head()#查看處理結果

0 01 12 13 14 0Name: Sex, dtype: int32

Embarked用啞變數進行定性轉換

all_data[Embarked]=pd.get_dummies(all_data[Embarked]).astype(int)

all_data.Embarked.head()#查看處理結果

0 01 12 03 04 0Name: Embarked, dtype: int32

（3）、Age特徵缺失值填充

all_data.info()#查看數據總覽，篩選用於隨機森林預測Age的特徵

<class pandas.core.frame.DataFrame>Int64Index: 1309 entries, 0 to 417Data columns (total 12 columns):PassengerId 1309 non-null int64Survived 1309 non-null int64Pclass 1309 non-null int64Name 1309 non-null objectSex 1309 non-null int32Age 1046 non-null float64SibSp 1309 non-null int64Parch 1309 non-null int64Ticket 1309 non-null objectFare 1309 non-null float64Cabin 1309 non-null int64Embarked 1309 non-null int32dtypes: float64(2), int32(2), int64(6), object(2)memory usage: 122.7+ KB

age_predict=all_data[[Age,Pclass,Sex,SibSp,Parch,Fare,Cabin]]#PassengerId、Name、Ticket捨棄，Survived不能用於預測，

age_predict.head()

age_predict_notnull=age_predict.loc[age_predict[Age].notnull()]#提提取非空部分用於訓練

age_predict_isnull=age_predict.loc[age_predict[Age].isnull()]#提取空值部分用於預測

age_predict_notnull_train_x=age_predict_notnull.values[:,1:]#因變數（除Age以外的特徵）

age_predict_notnull_train_y=age_predict_notnull.values[:,0]#自變數（Age非空值部分）

RF=RandomForestClassifier()#引入模型

RF.fit(age_predict_notnull_train_x,age_predict_notnull_train_y.astype(int))#擬合模型

RandomForestClassifier(bootstrap=True, class_weight=None, criterion=gini, max_depth=None, max_features=auto, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1, oob_score=False, random_state=None, verbose=0, warm_start=False)

age_predict=RF.predict(age_predict_isnull.values[:,1:].astype(int))#預測空值

all_data.loc[all_data[Age].isnull(),Age]=age_predict#賦予預測值

all_data.Age.isnull().sum()#檢驗填充結果

0

四、訓練模型

all_data=all_data.drop([PassengerId,Name,Ticket],axis=1)#丟失無用特徵

all_data_train=all_data[:691]#訓練集有891行，選取前691行用作訓練集

all_data_test=all_data[691:891]#餘下用於測試集

all_data_train_x=all_data_train.values[:,1:]#訓練集的自變數

all_data_train_y=all_data_train.values[:,0]#訓練集的因變數

利用網格搜索進行參數調優

from sklearn.grid_search import GridSearchCV#引入網格搜索

param_test1= {n_estimators:list(range(10,100,5)),min_samples_leaf:list(range(10,100,5))}#窮舉搜索的參數列表gsearch1= GridSearchCV(estimator = RandomForestClassifier(random_state=50), param_grid =param_test1, scoring=roc_auc,cv=5)#創建GridSearchCV

gsearch1.fit(all_data_train_x,all_data_train_y)

GridSearchCV(cv=5, error_score=raise, estimator=RandomForestClassifier(bootstrap=True, class_weight=None, criterion=gini, max_depth=None, max_features=auto, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1, oob_score=False, random_state=50, verbose=0, warm_start=False), fit_params={}, iid=True, n_jobs=1, param_grid={n_estimators: [10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95], min_samples_leaf: [10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95]}, pre_dispatch=2*n_jobs, refit=True, scoring=roc_auc, verbose=0)

gsearch1.grid_scores_[:5]#查看前5行分數列表，因為文章字數受限 - -！

[mean: 0.84748, std: 0.03112, params: {min_samples_leaf: 10, n_estimators: 10}, mean: 0.84980, std: 0.03086, params: {min_samples_leaf: 10, n_estimators: 15}, mean: 0.84728, std: 0.03051, params: {min_samples_leaf: 10, n_estimators: 20}, mean: 0.84856, std: 0.02947, params: {min_samples_leaf: 10, n_estimators: 25}, mean: 0.84979, std: 0.02855, params: {min_samples_leaf: 10, n_estimators: 30},

gsearch1.best_score_#查看最好成績

0.853754709825509

gsearch1.best_params_#查看最優參數

{min_samples_leaf: 10, n_estimators: 40}

clf=gsearch1.best_estimator_#基於網格搜索後獲得最優隨機森林模型

clf.fit(all_data_train_x,all_data_train_y)#使用最優模型重新進行訓練

RandomForestClassifier(bootstrap=True, class_weight=None, criterion=gini, max_depth=None, max_features=auto, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=10, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=40, n_jobs=1, oob_score=False, random_state=50, verbose=0, warm_start=False)

print(Test accuracy: %.3f % clf.score(all_data_test.values[:,1:],all_data_test.values[:,0]))#檢驗模型在測試集上的準確度，85.5%結果還是不錯的

Test accuracy: 0.855

五、運行預測

predict_data=all_data.values[891:]#提取需要預測的數據

predict_result=clf.predict(predict_data[:,1:]).astype(int)#執行預測

test_data[Survived]=predict_result#添加預測結果數據到test文件

test_data[[PassengerId,Survived]].to_csv(d:workRF_predict.csv,index=False)#保存文件

推薦閱讀：