非平行超平面分類器算法研究
發(fā)布時間：2018-08-28 09:29
【摘要】：非平行超平面分類器(nonparallel hyperplane classifier,NHC)分類方法是在傳統(tǒng)支持向量機(support vector machine,SVM)基礎(chǔ)上發(fā)展起來的一類新的機器學(xué)習(xí)方法。對于二分類問題,傳統(tǒng)SVM依據(jù)大間隔準(zhǔn)則尋找單一的分類超平面,而NHC分類方法通常要為每類樣本尋找一個最佳決策超平面,即一對非平行的分類超平面。在線性模式下,NHC分類方法對異或(XOR)問題有著顯著的分類能力。鑒于NHC分類方法的優(yōu)勢,目前已經(jīng)成為機器學(xué)習(xí)領(lǐng)域的研究熱點。然而,NHC分類方法是一類比較新的機器學(xué)習(xí)方法,在諸多方面尚不成熟、不完善,需要進一步的研究和改進。本文主要從提升分類性能、提高學(xué)習(xí)速度等方面對現(xiàn)有的NHC分類方法進行了深入系統(tǒng)地研究。具體研究內(nèi)容如下:1.對局部保持孿生支持向量機進行研究。針對現(xiàn)有NHC分類方法中沒有充分考慮訓(xùn)練樣本集內(nèi)在局部幾何結(jié)構(gòu)及其潛藏的分類信息,從而可能導(dǎo)致算法分類性能不佳的問題,將局部保持投影(locality preserving projections,LPP)的思想直接引入到NHC分類方法中,提出一種基于局部信息保持的孿生支持向量機(locality preserving twin SVM,LPTSVM)。為了能夠有效降低算法二次規(guī)劃求解的時間復(fù)雜度,LPTSVM通過類間近鄰圖選取少量的邊界樣本來構(gòu)造優(yōu)化問題的約束條件。對于LPTSVM算法中可能出現(xiàn)的奇異性問題,從理論上給出了一種基于主成分分析(principal component analysis,PCA)的降維方法。2.對非線性最小二乘投影孿生支持向量機及相應(yīng)的遞歸學(xué)習(xí)算法進行研究。針對線性最小二乘投影孿生支持向量機(least squares projection twin SVM,LSPTSVM)不能有效處理非線性分類情況的問題,采用核映射技術(shù)將原空間中的訓(xùn)練樣本映射到高維特征空間,在此基礎(chǔ)上提出一種非線性最小二乘投影孿生支持向量機(kernel based LSPTSVM,KLSPTSVM)。為進一步提高KLSPTSVM算法的非線性分類性能,同樣采用核映射技術(shù)將線性模式下的遞歸學(xué)習(xí)算法也推廣到非線性模式并與KLSPTSVM分類算法相結(jié)合,提出非線性模式下的遞歸KLSPTSVM分類方法。3.對魯棒的局部加權(quán)孿生支持向量機進行研究。針對局部加權(quán)孿生支持向量機(weighted twin SVM with local information,WLTSVM)算法不能充分刻畫類內(nèi)樣本之間相似性,訓(xùn)練效率偏低和對噪聲敏感的問題,提出一種魯棒的局部加權(quán)孿生支持向量機(robust WLTSVM,RWLTSVM)。RWLTSVM選用高斯核函數(shù)定義類內(nèi)近鄰圖的權(quán)值矩陣并在此基礎(chǔ)上生成樣本權(quán)重,能夠更好的刻畫類內(nèi)樣本對決策超平面的貢獻程度。為了降低優(yōu)化問題求解時間復(fù)雜度,RWLTSVM將WLTSVM算法中不等式約束改成等式約束并通過求解方程組方法獲得問題解析解。另外,RWLTSVM還在等式約束條件中考慮了相反類樣本的類內(nèi)權(quán)重,從而能夠更好的免疫于噪聲問題。4.對加權(quán)投影孿生支持向量機及其相應(yīng)的最小二乘版算法進行研究。針對投影孿生支持向量機(projection twin SVM,PTSVM)在優(yōu)化問題中沒有考慮類內(nèi)訓(xùn)練樣本之間相關(guān)性的問題,提出一種加權(quán)投影孿生支持向量機(weighted PTSVM,WPTSVM)。WPTSVM通過在類內(nèi)構(gòu)造近鄰圖并在此基礎(chǔ)上賦予樣本特定的權(quán)重,以此來突出樣本對決策面的貢獻程度,進而改善算法的分類性能。此外,WPTSVM在優(yōu)化問題的不等式約束中同樣考慮了樣本權(quán)重,使得算法能夠很好的免疫于噪聲問題。為了進一步降低WPTSVM算法的訓(xùn)練時間復(fù)雜度,使其能夠勝任大規(guī)模數(shù)據(jù)處理,提出最小二乘版加權(quán)投影孿生支持向量機(least squares WPTSVM,LSWPTSVM)。LSWPTSVM通過解方程組獲得問題的解析解而不是WPTSVM中的二次規(guī)劃求解。
[Abstract]:Nonparallel hyperplane classifier (NHC) is a new class of machine learning methods based on traditional support vector machine (SVM). For binary classification problems, the traditional SVM searches for a single classification hyperplane according to the large-interval criterion, while the NHC classification method is usually required. In linear mode, NHC classifier has a remarkable ability to classify XOR problems. Because of the advantages of NHC classifier, it has become a research hotspot in the field of machine learning. However, NHC classifier is a relatively new kind of machine. In this paper, the existing NHC classification methods are thoroughly and systematically studied from the aspects of improving classification performance and learning speed. The specific research contents are as follows: 1. Local Twin-Preserving Support Vector Machines are studied. Locality preserving projections (LPP) are directly introduced into NHC classification method, and a local information preserving twin is proposed. Locality preserving twin support vector machine (LPTSVM). In order to reduce the time complexity of quadratic programming, LPTSVM constructs the constraints of optimization problem by selecting a small number of boundary samples from the neighborhood graph between classes. For the singularity problem that may appear in LPTSVM algorithm, a Principal-based approach is proposed in theory. Principal component analysis (PCA) dimensionality reduction method. 2. Nonlinear least squares projection twin support vector machine (LSPTSVM) and its corresponding recursive learning algorithm are studied. Linear least squares projection twin support vector machine (LSPTSVM) can not effectively deal with the problem of nonlinear classification. The kernel-based LSPTSVM (KLSPTSVM) is proposed based on the mapping of training samples from the original space to the high-dimensional feature space. In order to further improve the nonlinear classification performance of the KLSPTSVM algorithm, the kernel-based LSPTSVM (KLSPTSVM) is also used to recursively implement the linear pattern. The recursive learning algorithm is also extended to the nonlinear model and combined with the KLSPTSVM classification algorithm. 3. The robust local weighted twin support vector machine (WLTSVM) is studied. The local weighted twin support vector machine (WLTSVM) algorithm is insufficient. In this paper, a robust local weighted twin support vector machine (RWLTSVM) is proposed to characterize the similarity between samples in a class, low training efficiency and noise sensitivity. RWLTSVM uses Gaussian kernel function to define the weighting matrix of the neighborhood graph in a class and generates the sample weights on the basis of it, which can better characterize the intra-class sample duels. In order to reduce the time complexity of solving optimization problems, RWLTSVM transforms inequality constraints into equality constraints and obtains analytical solutions by solving equations. In addition, RWLTSVM considers the intra-class weights of inverse class samples in equality constraints, so it is immune to noise better. 4. The weighted projection twin support vector machine (WPTSVM) and its corresponding least squares version algorithm are studied. A weighted projection twin support vector machine (WPTSVM) is proposed to optimize the projection twin support vector machine (PTSVM). WPTSVM emphasizes the contribution of the samples to the decision-making surface by constructing the nearest neighbor graph in the class and giving the samples specific weights on the basis of it, so as to improve the classification performance of the algorithm. In addition, WPTSVM also considers the sample weights in the inequality constraints of the optimization problem, so that the algorithm can be well immune to noise problems. To further reduce the training time complexity of WPTSVM algorithm and make it competent for large-scale data processing, least squares weighted projection twin support vector machine (LSWPTSVM) is proposed. LSWPTSVM obtains the analytical solution of the problem by solving the system of equations instead of the quadratic programming in WPTSVM.
【學(xué)位授予單位】：中國礦業(yè)大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2015
【分類號】：TP181;TP391.4
，
本文編號：2208988
[Abstract]:Nonparallel hyperplane classifier (NHC) is a new class of machine learning methods based on traditional support vector machine (SVM). For binary classification problems, the traditional SVM searches for a single classification hyperplane according to the large-interval criterion, while the NHC classification method is usually required. In linear mode, NHC classifier has a remarkable ability to classify XOR problems. Because of the advantages of NHC classifier, it has become a research hotspot in the field of machine learning. However, NHC classifier is a relatively new kind of machine. In this paper, the existing NHC classification methods are thoroughly and systematically studied from the aspects of improving classification performance and learning speed. The specific research contents are as follows: 1. Local Twin-Preserving Support Vector Machines are studied. Locality preserving projections (LPP) are directly introduced into NHC classification method, and a local information preserving twin is proposed. Locality preserving twin support vector machine (LPTSVM). In order to reduce the time complexity of quadratic programming, LPTSVM constructs the constraints of optimization problem by selecting a small number of boundary samples from the neighborhood graph between classes. For the singularity problem that may appear in LPTSVM algorithm, a Principal-based approach is proposed in theory. Principal component analysis (PCA) dimensionality reduction method. 2. Nonlinear least squares projection twin support vector machine (LSPTSVM) and its corresponding recursive learning algorithm are studied. Linear least squares projection twin support vector machine (LSPTSVM) can not effectively deal with the problem of nonlinear classification. The kernel-based LSPTSVM (KLSPTSVM) is proposed based on the mapping of training samples from the original space to the high-dimensional feature space. In order to further improve the nonlinear classification performance of the KLSPTSVM algorithm, the kernel-based LSPTSVM (KLSPTSVM) is also used to recursively implement the linear pattern. The recursive learning algorithm is also extended to the nonlinear model and combined with the KLSPTSVM classification algorithm. 3. The robust local weighted twin support vector machine (WLTSVM) is studied. The local weighted twin support vector machine (WLTSVM) algorithm is insufficient. In this paper, a robust local weighted twin support vector machine (RWLTSVM) is proposed to characterize the similarity between samples in a class, low training efficiency and noise sensitivity. RWLTSVM uses Gaussian kernel function to define the weighting matrix of the neighborhood graph in a class and generates the sample weights on the basis of it, which can better characterize the intra-class sample duels. In order to reduce the time complexity of solving optimization problems, RWLTSVM transforms inequality constraints into equality constraints and obtains analytical solutions by solving equations. In addition, RWLTSVM considers the intra-class weights of inverse class samples in equality constraints, so it is immune to noise better. 4. The weighted projection twin support vector machine (WPTSVM) and its corresponding least squares version algorithm are studied. A weighted projection twin support vector machine (WPTSVM) is proposed to optimize the projection twin support vector machine (PTSVM). WPTSVM emphasizes the contribution of the samples to the decision-making surface by constructing the nearest neighbor graph in the class and giving the samples specific weights on the basis of it, so as to improve the classification performance of the algorithm. In addition, WPTSVM also considers the sample weights in the inequality constraints of the optimization problem, so that the algorithm can be well immune to noise problems. To further reduce the training time complexity of WPTSVM algorithm and make it competent for large-scale data processing, least squares weighted projection twin support vector machine (LSWPTSVM) is proposed. LSWPTSVM obtains the analytical solution of the problem by solving the system of equations instead of the quadratic programming in WPTSVM.
【學(xué)位授予單位】：中國礦業(yè)大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2015
【分類號】：TP181;TP391.4
，
本文編號：2208988
本文鏈接：http://www.sikaile.net/shoufeilunwen/xxkjbs/2208988.html
最近更新
教材專著
