前期准备

导入一些必要库

1
2
3

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

查看数据

1
2
3

df.head()

df.isnull().sum()

	工龄	薪水
0	0.0	10808
1	0.1	13611
2	0.2	12306
3	0.3	12151
4	0.4	13057

工龄 0
薪水 0
dtype: int64

1 2	plt.scatter(df['工龄'], df['薪水']) plt.show()

plt.figure(figsize=(10, 4))
plt.subplot(1,2,1)
plt.boxplot(df['薪水'])

plt.subplot(1,2,2)
plt.boxplot(df['工龄'])
plt.show()

发现数据无异常值和缺失值，接下来我们直接使用各种模型进行应用

拆分数据

from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score

x_train, x_test, y_train, y_test = train_test_split(df[['工龄']], df['薪水'], test_size=0.2, random_state=42)

线性回归

from sklearn.linear_model import LinearRegression

lr = LinearRegression()
lr.fit(x_train, y_train)
y_pred = lr.predict(x_test)

print('mse:', mean_squared_error(y_test, y_pred))
print('r2:', r2_score(y_test, y_pred))

mse: 4855442.658615259
r2: 0.8815705091563912

随机森林

from sklearn.ensemble import RandomForestRegressor

rm = RandomForestRegressor()
rm.fit(x_train, y_train)
y_pred = rm.predict(x_test)

print('mse:', mean_squared_error(y_test, y_pred))
print('r2:', r2_score(y_test, y_pred))

mse: 2389839.0208794717
r2: 0.9417092449977235

XGBoost

from xgboost import XGBRegressor

xgb = XGBRegressor(random_state=42)
xgb.fit(x_train, y_train)
y_pred_xgb = xgb.predict(x_test)

print("MSE:", mean_squared_error(y_test, y_pred_xgb))
print("r2:", r2_score(y_test, y_pred_xgb))

MSE: 3569750.747018099
r2: 0.9129299235657095

LGBM

from lightgbm import LGBMRegressor

lgb = LGBMRegressor(random_state=42)
lgb.fit(x_train, y_train)
y_pred_lgb = lgb.predict(x_test)

print("MSE:", mean_squared_error(y_test, y_pred_lgb))
print("r2:", r2_score(y_test, y_pred_lgb))

MSE: 3749728.9875734868
r2: 0.9085400598827179

SVM

from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVR
from sklearn.pipeline import Pipeline

svr_pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('svr', SVR(kernel='rbf', C=100, gamma='scale'))
])

svr_pipeline.fit(x_train, y_train)
y_pred = svr_pipeline.predict(x_test)

print("MSE:", mean_squared_error(y_test, y_pred))
print("r2:", r2_score(y_test, y_pred))

MSE: 19519338.352298636
r2: 0.5239022546038833
发现其效果并不好 SVM需要进行调参使用网格优化

from sklearn.model_selection import GridSearchCV

param_grid = {
    'svr__C': [0.1, 1, 10, 100, 1000],
    'svr__gamma': ['scale', 'auto', 0.001, 0.01, 0.1, 1],
    'svr__kernel': ['rbf', 'poly', 'linear']
}

grid = GridSearchCV(svr_pipeline, param_grid, cv=5, scoring='r2', n_jobs=-1)
grid.fit(x_train, y_train)

print("最佳参数:", grid.best_params_)
print("最佳交叉验证r2:", grid.best_score_)

y_pred = grid.predict(x_test)

最佳参数: {‘svr__C’: 1000, ‘svr__gamma’: ‘scale’, ‘svr__kernel’: ‘rbf’}
最佳交叉验证r2: 0.8533151968605445

汇总

上述五种模型的成绩为

模型	r2
线性回归	0.881
随机森林	0.941
XGBoost	0.912
LGBM	0.908
SVM	0.853

我们看到面对同一份数据不同的模型其成绩各不相同
在使用线性回归的时候我们通常假定变量和目标之间存在线性关系即y=kx 当变量和目标之间的确是存在线性关系时模型得分会很高若变量和目标之间不是线性关系例如 y=kx^2 那线性回归的得分肯定不高
线性回归的特点是简单复杂性低泛化稳定但容易欠拟合

随机森林属于集成算法训练大量决策树每颗树用不同的有放回抽烟者每次分裂时在随机子集特征上寻找最优划分回归任务就取树预测的平均值
随机森林的特点是鲁棒性好对异常值不敏感免交叉验证对低维数据表现有限

XGBoost是基于梯度提升对思想逐步构建决策树每棵树拟合前一轮的残差采用二阶泰勒展开来加速优化加入正则化项控制复杂度
XGBoost的特点是准确率高适用性强超参数多调参麻烦

LGBM是一种高效的梯度提升框架
LGBM的特点是训练和预测速度快内存占用小适合稀疏数据容易过拟合

SVM原理是在高维空间中找到一个最优超平面 使得不同类别的样本间隔最大在回归时使预测函数在一定容忍范围内尽量平滑
SVM的特点是泛化能力强适合高维小样本核函数灵活计算复杂度高参数敏感可解释性差

智浩的Blog

商业数据分析--IT行业收入表