3.1

式3.2 $f(x)=\omega ^{T}x+b$ 中，$\omega ^{T}$ 和b有各自的意义，简单来说，$\omega ^{T}$ 决定学习得到模型(直线、平面)的方向，而b则决定截距，当学习得到的模型恰好经过原点时，可以不考虑偏置项b。偏置项b实质上就是体现拟合模型整体上的浮动，可以看做是其它变量留下的偏差的线性修正，因此一般情况下是需要考虑偏置项的。但如果对数据集进行了归一化处理，即对目标变量减去均值向量，此时就不需要考虑偏置项了。

3.2

对区间[a,b]上定义的函数f(x)，若它对区间中任意两点x1，x2均有$f(\frac{x1+x2}{2})\leq \frac{f(x1)+f(x2)}{2}$，则称f(x)为区间[a,b]上的凸函数。对于实数集上的函数，可通过二阶导数来判断：若二阶导数在区间上非负，则称为凸函数，在区间上恒大于零，则称为严格凸函数。

对于式3.18 $y=\frac{1}{1+e^{-(\omega ^{T}x+b)}}$，有

$\frac{dy}{d\omega ^{T}}=\frac{1}{(1+e^{-(\omega ^{T}x+b)})^{2}}e^{-(\omega ^{T}x+b)}(-x)=(-x)\frac{1}{1+e^{-(\omega ^{T}x+b)}}(1-\frac{1}{1+e^{-(\omega ^{T}x+b)}})=xy(y-1)=x(y^{2}-y)$

$\frac{d}{d\omega ^{T}}(\frac{dy}{d\omega ^{T}})=x(2y-1)(\frac{dy}{d\omega ^{T}})=x^{2}y(2y-1)(y-1)$

其中，y的取值范围是(0,1)，不难看出二阶导有正有负，所以该函数非凸。

3.3

对率回归即Logis regression

西瓜集数据如图所示：

将好瓜这一列变量用0/1变量代替，进行对率回归学习，python代码如下：

import numpy as npimport matplotlib.pyplot as pltimport pandas as pdfrom sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好瓜']good_melon = dataset[dataset['好瓜'] == 1]bad_melon = dataset[dataset['好瓜'] == 0]#画图f1 = plt.figure(1)plt.title('watermelon_3a')plt.xlabel('density')plt.ylabel('radio_sugar')plt.xlim(0,1)plt.ylim(0,1)plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')plt.legend(loc='upper right')#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred, target_names=['Bad','Good']))print(log_model.coef_)theta1, theta2 = log_model.coef_[0][0], log_model.coef_[0][1]X_pred = np.linspace(0,1,100)line_pred = theta1 + theta2 * X_predplt.plot(X_pred, line_pred)plt.show()

import numpy as npimport matplotlib.pyplot as pltimport pandas as pdfrom sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好瓜']good_melon = dataset[dataset['好瓜'] == 1]bad_melon = dataset[dataset['好瓜'] == 0]#画图f1 = plt.figure(1)plt.title('watermelon_3a')plt.xlabel('density')plt.ylabel('radio_sugar')plt.xlim(0,1)plt.ylim(0,1)plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')plt.legend(loc='upper right')#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred))print(log_model.coef_)theta1, theta2 = log_model.coef_[0][0], log_model.coef_[0][1]X_pred = np.linspace(0,1,100)line_pred = theta1 + theta2 * X_predplt.plot(X_pred, line_pred)plt.show()

模型效果输出(查准率、查全率、预测效果评分)：

precision    recall  f1-score   support         Bad       0.75      0.60      0.67         5        Good       0.60      0.75      0.67         4     micro avg       0.67      0.67      0.67         9   macro avg       0.68      0.68      0.67         9weighted avg       0.68      0.67      0.67         9

也可以输出验证集的实际结果和预测结果：

密度    含糖率   Y_test    Y_pred1     0.774  0.376    1         16     0.481  0.149    1         08     0.666  0.091    0         09     0.243  0.267    0         113    0.657  0.198    0         04     0.556  0.215    1         12     0.634  0.264    1         114    0.360  0.370    0         110    0.245  0.057    0         0

3.4

首先附上使用葡萄酒品质数据做的对率回归学习代码

import numpy as npimport matplotlib.pyplot as pltimport pandas as pdpd.set_option('display.max_rows',None)pd.set_option('max_colwidth',200)pd.set_option('expand_frame_repr', False)from sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/winequality-red_new.csv')#数据预处理dataset['quality2'] = dataset['quality'].apply(lambda x: 0 if x < 5 else 1)    #新加入二分类变量是否为好酒，基于原数据中quality的值，其大于等于5就定义为好酒，反之坏酒X = dataset[["fixed_acidity","volatile_acidity","citric_acid","residual_sugar","chlorides","free_sulfur_dioxide","total_sulfur_dioxide","density","pH","sulphates","alcohol"]]Y = dataset["quality2"]#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred))print(log_model.coef_)

其中，从UCI下载的数据集格式有问题，无法直接使用，先编写程序将格式调整完毕再使用数据

fr = open('/home/zwt/Desktop/winequality-red.csv','r',encoding='utf-8')fw = open('/home/zwt/Desktop/winequality-red_new.csv','w',encoding='utf-8')f = fr.readlines()for line in f:    line = line.replace(';',',')    fw.write(line)fr.close()fw.close()

两种方法的错误率比较

from sklearn.linear_model import LogisticRegressionfrom sklearn import model_selectionfrom sklearn.datasets import load_wine# 载入wine数据dataset = load_wine()#10次10折交叉验证法生成训练集和测试集def tenfolds():    k = 0    truth = 0    while k < 10:        kf = model_selection.KFold(n_splits=10, random_state=None, shuffle=True)        for x_train_index, x_test_index in kf.split(dataset.data):            x_train = dataset.data[x_train_index]            y_train = dataset.target[x_train_index]            x_test = dataset.data[x_test_index]            y_test = dataset.target[x_test_index]        # 用对率回归进行训练，拟合数据        log_model = LogisticRegression()        log_model.fit(x_train, y_train)        # 用训练好的模型预测        y_pred = log_model.predict(x_test)        for i in range(len(x_test)):         #这里和留一法不同，是因为10折交叉验证的验证集是len(dataset.target)/10，验证集的预测集也是，都是一个列表，是一串数字，而留一法是一个数字            if y_pred[i] == y_test[i]:                truth += 1        k += 1    # 计算精度    accuracy = truth/(len(x_train)+len(x_test))  #accuracy = truth/len(dataset.target)    print("用10次10折交叉验证对率回归的精度是：", accuracy)tenfolds()#留一法def leaveone():    loo = model_selection.LeaveOneOut()    i = 0    true = 0    while i < len(dataset.target):        for x_train_index, x_test_index in loo.split(dataset.data):            x_train = dataset.data[x_train_index]            y_train = dataset.target[x_train_index]            x_test = dataset.data[x_test_index]            y_test = dataset.target[x_test_index]        # 用对率回归进行训练，拟合数据        log_model = LogisticRegression()        log_model.fit(x_train, y_train)        # 用训练好的模型预测        y_pred = log_model.predict(x_test)        if y_pred == y_test:            true += 1        i += 1    # 计算精度    accuracy = true / len(dataset.target)    print("用留一法验证对率回归的精度是：", accuracy)leaveone()

3.5

import numpy as npimport pandas as pdimport matplotlib.pyplot as pltfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysisfrom sklearn import model_selectionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好瓜']#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练LDA_model = LinearDiscriminantAnalysis()LDA_model.fit(X_train,Y_train)#验证Y_pred = LDA_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred, target_names=['Bad','Good']))print(LDA_model.coef_)#画图good_melon = dataset[dataset['好瓜'] == 1]bad_melon = dataset[dataset['好瓜'] == 0]plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')

import numpy as npimport matplotlib.pyplot as pltdata = [[0.697, 0.460, 1],        [0.774, 0.376, 1],        [0.634, 0.264, 1],        [0.608, 0.318, 1],        [0.556, 0.215, 1],        [0.403, 0.237, 1],        [0.481, 0.149, 1],        [0.437, 0.211, 1],        [0.666, 0.091, 0],        [0.243, 0.267, 0],        [0.245, 0.057, 0],        [0.343, 0.099, 0],        [0.639, 0.161, 0],        [0.657, 0.198, 0],        [0.360, 0.370, 0],        [0.593, 0.042, 0],        [0.719, 0.103, 0]]#数据集按瓜好坏分类data = np.array([i[:-1] for i in data])X0 = np.array(data[:8])X1 = np.array(data[8:])#求正反例均值miu0 = np.mean(X0, axis=0).reshape((-1, 1))miu1 = np.mean(X1, axis=0).reshape((-1, 1))#求协方差cov0 = np.cov(X0, rowvar=False)cov1 = np.cov(X1, rowvar=False)#求出wS_w = np.mat(cov0 + cov1)Omiga = S_w.I * (miu0 - miu1)#画出点、直线plt.scatter(X0[:, 0], X0[:, 1], c='b', label='+', marker = '+')plt.scatter(X1[:, 0], X1[:, 1], c='r', label='-', marker = '_')plt.plot([0, 1], [0, -Omiga[0] / Omiga[1]], label='y')plt.xlabel('密度', fontproperties='SimHei', fontsize=15, color='green');plt.ylabel('含糖率', fontproperties='SimHei', fontsize=15, color='green');plt.title(r'LinearDiscriminantAnalysis', fontproperties='SimHei', fontsize=25);plt.legend()plt.show()

3.6

对于非线性可分的数据，要想使用判别分析，一般思想是将其映射到更高维的空间上，使它在高维空间上线性可分进一步使用判别分析。

3.7

3.8

理论上的(纠错输出码)ECOC码能理想纠错的重要条件是每个码位出错的概率相当，因为如果某个码位的错误率很高，会导致这位始终保持相同的结果，不再有分类作用，这就相当于全0或者全 1的分类器。

3.9

书中提到，对于OvR，MvM来说，由于对每个类进行了相同的处理，其拆解出的二分类任务中类别不平衡的影响会相互抵消，因此通常不需要专门处理。以ECOC编码为例，每个生成的二分类器会将所有样本分成较为均衡的二类，使类别不平衡的影响减小。当然拆解后仍然可能出现明显的类别不平衡现象，比如一个超级大类和一群小类。

3.10

 

 

（密码rw81）