35丨AdaBoost（下）：如何使用AdaBoost对房价进行预测？

源代码中： # 从 12000 个数据中取前 2000 行作为测试集，其余作为训练集 test_x, test_y = X[2000:],y[2000:] train_x, train_y = X[:2000],y[:2000] 这个部分的代码写错了吧 应该是： test_x, test_y = x[: 2000], y[: 2000] train_x, train_y = x[2000:], y[2000:]

结果仍然为AdaBoost算法最优。 个人发现，前两个分类器出结果很快 分析最优： 1.AdaBoost算法经过了更多运算，特别是在迭代弱分类器和组合上 2.良好组合起来的个体，能够创造更大的价值。 决策树弱分类器准确率为 0.7867 决策树分类器准确率为 0.7891 AdaBoost 分类器准确率为 0.8138 import numpy as np import pandas as pd from sklearn.model_selection import cross_val_score from sklearn.tree import DecisionTreeClassifier from sklearn.ensemble import AdaBoostClassifier from sklearn.feature_extraction import DictVectorizer # 1.数据加载 train_data=pd.read_csv('./Titanic_Data/train.csv') test_data=pd.read_csv('./Titanic_Data/test.csv') # 2.数据清洗 # 使用平均年龄来填充年龄中的 NaN 值 train_data['Age'].fillna(train_data['Age'].mean(),inplace=True) test_data['Age'].fillna(test_data['Age'].mean(),inplace=True) # 均价填充 train_data['Fare'].fillna(train_data['Fare'].mean(),inplace=True) test_data['Fare'].fillna(test_data['Fare'].mean(),inplace=True) # 使用登陆最多的港口来填充 train_data['Embarked'].fillna('S',inplace=True) test_data['Embarked'].fillna('S',inplace=True) # 特征选择 features=['Pclass','Sex','Age','SibSp','Parch','Fare','Embarked'] train_features=train_data[features] train_labels=train_data['Survived'] test_features=test_data[features] # 将符号化的Embarked对象抽象处理成0/1进行表示 dvec=DictVectorizer(sparse=False) train_features=dvec.fit_transform(train_features.to_dict(orient='record')) test_features=dvec.transform(test_features.to_dict(orient='record')) # 决策树弱分类器 dt_stump = DecisionTreeClassifier(max_depth=1,min_samples_leaf=1) dt_stump.fit(train_features, train_labels) print(u'决策树弱分类器准确率为 %.4lf' % np.mean(cross_val_score(dt_stump, train_features, train_labels, cv=10))) # 决策树分类器 dt = DecisionTreeClassifier() dt.fit(train_features, train_labels) print(u'决策树分类器准确率为 %.4lf' % np.mean(cross_val_score(dt, train_features, train_labels, cv=10))) # AdaBoost 分类器 ada = AdaBoostClassifier(base_estimator=dt_stump,n_estimators=200) ada.fit(train_features, train_labels) print(u'AdaBoost 分类器准确率为 %.4lf' % np.mean(cross_val_score(ada, train_features, train_labels, cv=10)))

由于乘客测试集缺失真实值，采用 K 折交叉验证准确率 -------------------- 运行结果： 决策树弱分类器准确率为 0.7867 决策树分类器准确率为 0.7813 AdaBoost 分类器准确率为 0.8138 ------------------------- 代码： import numpy as np from sklearn.tree import DecisionTreeClassifier from sklearn.ensemble import AdaBoostClassifier import pandas as pd from sklearn.feature_extraction import DictVectorizer from sklearn.model_selection import cross_val_score # 设置 AdaBoost 迭代次数 n_estimators=200 # 数据加载 train_data=pd.read_csv('./Titanic_Data/train.csv') test_data=pd.read_csv('./Titanic_Data/test.csv') # 模块 2：数据清洗 # 使用平均年龄来填充年龄中的 NaN 值 train_data['Age'].fillna(train_data['Age'].mean(),inplace=True) test_data['Age'].fillna(test_data['Age'].mean(),inplace=True) # 使用票价的均值填充票价中的 nan 值 train_data['Fare'].fillna(train_data['Fare'].mean(),inplace=True) test_data['Fare'].fillna(test_data['Fare'].mean(),inplace=True) # 使用登录最多的港口来填充登录港口Embarked的 nan 值 train_data['Embarked'].fillna('S',inplace=True) test_data['Embarked'].fillna('S',inplace=True) # 特征选择 features=['Pclass','Sex','Age','SibSp','Parch','Fare','Embarked'] train_features=train_data[features] train_labels=train_data['Survived'] test_features=test_data[features] # 将符号化的Embarked对象处理成0/1进行表示 dvec=DictVectorizer(sparse=False) train_features=dvec.fit_transform(train_features.to_dict(orient='record')) test_features=dvec.transform(test_features.to_dict(orient='record')) # 决策树弱分类器 dt_stump = DecisionTreeClassifier(max_depth=1,min_samples_leaf=1) dt_stump.fit(train_features, train_labels) print(u'决策树弱分类器准确率为 %.4lf' % np.mean(cross_val_score(dt_stump, train_features, train_labels, cv=10))) # 决策树分类器 dt = DecisionTreeClassifier() dt.fit(train_features, train_labels) print(u'决策树分类器准确率为 %.4lf' % np.mean(cross_val_score(dt, train_features, train_labels, cv=10))) # AdaBoost 分类器 ada = AdaBoostClassifier(base_estimator=dt_stump,n_estimators=n_estimators) ada.fit(train_features, train_labels) print(u'AdaBoost 分类器准确率为 %.4lf' % np.mean(cross_val_score(ada, train_features, train_labels, cv=10)))

跑第二块代码是需要引入两个模块 from sklearn.tree import DecisionTreeRegressor from sklearn.neighbors import KNeighborsRegressor

ax = fig.add_subplot(111)ax.plot([1,n_estimators],[dt_stump_err]*2, 'k-', label=u'决策树弱分类器 错误率')ax.plot([1,n_estimators],[dt_err]*2,'k--', label=u'决策树模型 错误率')ada_err = np.zeros((n_estimators,)). 疑问：这里*2是什么意思，能解析下代码吗？

得到结果： CART决策树K折交叉验证准确率: 0.39480897860892333 AdaBoostK折交叉验证准确率: 0.4376641797318339 from sklearn.tree import DecisionTreeRegressor from sklearn.ensemble import AdaBoostRegressor from sklearn.feature_extraction import DictVectorizer from sklearn.model_selection import cross_val_predict import pandas as pd import numpy as np #读取数据 path = '/Users/apple/Desktop/GitHubProject/Read mark/数据分析/geekTime/data/' train_data = pd.read_csv(path + 'Titannic_Data_train.csv') test_data = pd.read_csv(path + 'Titannic_Data_test.csv') #数据清洗 train_data['Age'].fillna(train_data['Age'].mean(),inplace=True) test_data['Age'].fillna(test_data['Age'].mean(), inplace=True) train_data['Embarked'].fillna('S', inplace=True) test_data['Embarked'].fillna('S', inplace=True) #特征选择 features = ['Pclass','Sex','Age','SibSp','Parch','Embarked'] train_features = train_data[features] train_result = train_data['Survived'] test_features = test_data[features] devc = DictVectorizer(sparse=False) train_features = devc.fit_transform(train_features.to_dict(orient='record')) test_features = devc.fit_transform(test_features.to_dict(orient='record')) #构造决策树，进行预测 tree_regressor = DecisionTreeRegressor() tree_regressor.fit(train_features,train_result) predict_tree = tree_regressor.predict(test_features) #交叉验证准确率 print('CART决策树K折交叉验证准确率:', np.mean(cross_val_predict(tree_regressor,train_features,train_result,cv=10))) #构造AdaBoost ada_regressor = AdaBoostRegressor() ada_regressor.fit(train_features,train_result) predict_ada = ada_regressor.predict(test_features) #交叉验证准确率 print('AdaBoostK折交叉验证准确率:',np.mean(cross_val_predict(ada_regressor,train_features,train_result,cv=10)))

弱分类器准确率为 0.7868 决策树分类器准确率为 0.7823 AdaBoost分类器准确率为:0.8115 #!/usr/bin/env python # -*- coding:utf-8 -*- # Author:Peter import numpy as np import pandas as pd from sklearn.ensemble import AdaBoostClassifier from sklearn.feature_extraction import DictVectorizer from sklearn.model_selection import cross_val_score from sklearn.tree import DecisionTreeClassifier # 迭代次数 n_estimators = 200 train_data = pd.read_csv(r'data/Titanic_Data_train.csv') test_data = pd.read_csv(r'data/Titanic_Data_Test.csv') # 用平均年龄将缺失的年龄补齐 train_data['Age'].fillna(train_data['Age'].mean(), inplace=True) test_data['Age'].fillna(test_data['Age'].mean(), inplace=True) # 用平均票价将缺失的票价补齐 train_data['Fare'].fillna(train_data['Fare'].mean(), inplace=True) test_data['Fare'].fillna(test_data['Fare'].mean(), inplace=True) # 用登船港口最多的S补齐缺失 train_data['Embarked'].fillna('S', inplace=True) test_data['Embarked'].fillna('S', inplace=True) # 将可用来分类的数据放到训练集中 features = ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked'] train_features = train_data[features] train_labels = train_data['Survived'] test_features = test_data[features] # 字符串数据规范化，转为int型 dvec = DictVectorizer(sparse=False) train_features = dvec.fit_transform(train_features.to_dict(orient='record')) test_features = dvec.transform(test_features.to_dict(orient='record')) # 弱分类器 dt_stump = DecisionTreeClassifier(max_depth=1, min_samples_leaf=1) dt_stump.fit(train_features, train_labels) print(u'弱分类器准确率为 %.4lf' % dt_stump.score(train_features, train_labels)) # 决策树分类器 dt = DecisionTreeClassifier() dt.fit(train_features, train_labels) print(u'决策树分类器准确率为 %.4lf' % np.mean(cross_val_score(dt, train_features, train_labels, cv=10))) # AdaBoost分类器 ada = AdaBoostClassifier(base_estimator=dt_stump, n_estimators=n_estimators) ada.fit(train_features, train_labels) ada_score = np.mean(cross_val_score(ada, train_features, train_labels, cv=10)) print("AdaBoost分类器准确率为:%.4lf" % ada_score)

在AdaBoost、决策树回归、KNN房价预测对比中发现，随机种子对决策树的预测结果有影响。 分别测试了三种不同的随机种子： dec_regressor=DecisionTreeRegressor(random_state=1) dec_regressor=DecisionTreeRegressor(random_state=20) dec_regressor=DecisionTreeRegressor(random_state=30) 测试结果为： 决策树均方误差1 = 36.65 决策树均方误差20 = 25.54 决策树均方误差30 = 37.19 思考： 此处考虑这里没有限制种子的随机性，对比的结果可能过于随机了，无法真实反映算法效果，两种算法原理中随机种子的应用情况不同。思考是不是采用多次随机MSE结果求平均的方法作为【比较项】更为合适 KNN算法无随机种子影响。

使用自带的数据集就不用做数据规范化么？

老师，请问AdaBoost模型在预测前需不需要对数据进行标准化或者归一化，做有什么好处，不做有什么好处呢