基于不確定理論的云數(shù)據(jù)處理關鍵技術研究
發(fā)布時間：2019-06-18 18:05
【摘要】：2016年1月,RightScale對全球1000多個企業(yè)用戶進行了關于公有云、私有云和混合云的使用情況調查,該調查報告結果顯示95%的受訪者正在使用云�，F(xiàn)實世界中,不確定因素普遍存在于各種現(xiàn)象中。在云計算環(huán)境下,云數(shù)據(jù)中心中云數(shù)據(jù)、虛擬機的遷移、調度等問題都具有不確定性。對于不確定性數(shù)據(jù)處理,目前已有很多成果,多集中在實體數(shù)據(jù)的不確定性,對現(xiàn)實中一些實際問題覆蓋還不夠。對于實體間關系的非確定性處理,已有文獻運用隨機和模糊理論解決近鄰查詢處理問題。而實體間關系有時還表現(xiàn)為主觀不確定性,這種主觀不確定性既不是隨機的也不是模糊的�，F(xiàn)實中,很多問題無法獲得歷史數(shù)據(jù),從而無法用概率論求解事件發(fā)生的頻率,此時必須依據(jù)專家經(jīng)驗對事件可能發(fā)生的信度進行評估,此方法使得信度的方差遠遠大于頻率。為了處理云數(shù)據(jù)的主觀不確定性,將采用不確定理論對云數(shù)據(jù)的處理技術展開研究。本文致力于云數(shù)據(jù)查詢處理、查詢優(yōu)化關鍵技術的研究,由于異構性、隱私性、隱私保護、數(shù)據(jù)不完整、數(shù)據(jù)不精確等原因,云數(shù)據(jù)中心的數(shù)據(jù)存在不確定性,借鑒和吸收不確定理論的相關研究,將云數(shù)據(jù)中心抽象為不確定圖。根據(jù)不確定圖的路徑查詢算法,對云數(shù)據(jù)的查詢處理、查詢優(yōu)化進行深入的探討,本文的主要工作和貢獻可以歸納為：(1)提出了云數(shù)據(jù)安全防護框架。該框架主要包括物理安全、虛擬網(wǎng)絡安全、云操作系統(tǒng)安全、虛擬集群安全、數(shù)據(jù)安全、SaaS/PaaS/IaaS安全、安全管理與安全運維等層次模塊。該框架在安全目標、系統(tǒng)資源類型、基礎安全技術方面與傳統(tǒng)安全是相同的,而又有其特有的安全問題,主要包括：虛擬化安全問題和與云計算分租服務模式相關的一些安全問題。該框架在虛擬化安全、數(shù)據(jù)安全和隱私保護等方面具有更好的安全性和保護能力。(2)提出了基于云數(shù)據(jù)安全防護框架的不確定隨機故障樹風險分析方法。該方法基于不確定理論和機會理論,對故障樹進行構建和分析。故障樹由基于底事件之間的邏輯關系構成。若底事件的故障率由歷史數(shù)據(jù)獲得,則被表征為隨機變量：若沒有歷史數(shù)據(jù),但可從專家主觀判斷得到,則被表征為不確定變量。除此之外事件發(fā)生的機會是不確定的隨機變量,因此構建了混合仿真算法來計算頂事件發(fā)生的機會。通過不確定隨機故障樹分析法對所提出的云數(shù)據(jù)安全防護框架進行風險分析。(3)提出了不確定網(wǎng)絡條件可信近鄰查詢方法。該方法包括可信距離的計算(CMCD)算法,可達路徑長度計算(CMFP)算法,可達路徑期望長度計算(CMDLFP)算法,條件可信k近鄰查詢(QMCCK)算法。將不確定網(wǎng)絡建模為不確定賦權圖,定義不確定圖的樣本圖,樣本圖指數(shù),基礎網(wǎng)絡,可達路徑長度及可達路徑期望長度,并給出基于不確定理論的高效不確定條件可信近鄰查詢算法。將不確定網(wǎng)絡上的近鄰查詢等價地轉化為基礎網(wǎng)絡上的近臨查詢問題。該可信近鄰查詢算法能夠從非確定角度解決不確定網(wǎng)絡環(huán)境下的近鄰查詢問題。(4)提出了基于不確定理論的不確定性數(shù)據(jù)Top-k查詢算法。將不確定性數(shù)據(jù)集中的元組建模為不確定網(wǎng)絡,將有序元組的Top-k查詢等價轉化為相應樣本圖中邊的不確定測度關系,并對樣本圖依據(jù)所包含邊的排序位置進行分類,該算法避免計算所有元組在樣本圖中的排名不確定測度值,提高了不確定性數(shù)據(jù)的Top-k查詢計算效率。將不確定性數(shù)據(jù)中,基于參數(shù)化排名函數(shù)的Top-k查詢等價轉換為依Top-k值不同的有限查詢,并結合Spark Map-Reduce編程框架完成了系統(tǒng)實現(xiàn)。
[Abstract]:In January 2016, the RightScale conducted a survey of the use of public clouds, private clouds and hybrid clouds for more than 1,000 enterprise users worldwide, and the survey found that 95% of the respondents were using the cloud. In the real world, uncertainty is common in various phenomena. In the cloud computing environment, the cloud data in the cloud data center, the migration and the scheduling of the virtual machine and the like have the uncertainty. There are many achievements in the data processing of the uncertainty, and the uncertainty of the entity's data is not enough to cover some of the real problems in the real world. For the non-deterministic processing of the relation between the entities, the existing literature uses the random and fuzzy theory to solve the problem of neighbor query processing. The relationship between the entities is sometimes also subjective, and the subjective uncertainty is neither random nor fuzzy. In reality, many problems can't get the historical data, so we can not use the probability theory to solve the frequency of the event. At this time, it is necessary to evaluate the reliability of the event based on the experience of the experts, which makes the variance of the reliability far greater than the frequency. In order to deal with the subjective uncertainty of cloud data, the process technology of cloud data will be studied with the uncertainty theory. This paper is devoted to the research of the key technology of cloud data query processing and query optimization. Because of the heterogeneity, privacy, privacy protection, incomplete data and inaccurate data, the data of the cloud data center is uncertain, and the relevant research of the uncertainty theory is used for reference and absorption. The cloud data center is abstracted as an uncertainty diagram. According to the path query algorithm of the uncertain graph, the query processing and query optimization of the cloud data are discussed in-depth. The main work and contribution of this paper can be summarized as follows: (1) The cloud data safety protection framework is proposed. The framework mainly includes physical security, virtual network security, cloud operating system security, virtual cluster security, data security, SaaS/ PaaS/ IaaS security, security management and security operation and maintenance level modules. The framework is the same as the traditional security in the aspects of security objective, system resource type and basic security technology, but also has the special security problem, mainly including: the virtualization security problem and some safety problems related to the cloud computing sublease service mode. The framework has better security and protection capabilities in terms of virtualization security, data security, and privacy protection. (2) The risk analysis method for uncertain random fault tree based on cloud data security protection framework is presented. The method is based on the theory of uncertain theory and opportunity, and the fault tree is constructed and analyzed. The fault tree is composed of a logical relationship based on the bottom event. If the failure rate of the bottom event is obtained from the historical data, it is characterized as a random variable: if there is no historical data, it can be obtained from the subjective judgment of the expert and is characterized as an uncertain variable. In addition, the chance of the occurrence of the event is an uncertain random variable, so a hybrid simulation algorithm is constructed to calculate the opportunity for the top event to occur. The proposed cloud data safety protection framework is analyzed by uncertain stochastic fault tree analysis. And (3) the method for querying the trusted neighbor of the network condition is proposed. The method comprises a CMDCD algorithm, a reachable path length calculation (CMDFP) algorithm, a reachable path expectation length calculation (CMDLFP) algorithm, and a conditional trusted k-neighbor query (QMCCK) algorithm. The uncertain network is modeled as an uncertain weight graph, a sample graph, a sample map index, a basic network, a reachable path length and a reachable path expectation length of the uncertain graph are defined, and an efficient and uncertain conditional trusted neighbor query algorithm based on the uncertainty theory is given. The neighbor query on the network is not determined to be equivalently converted into a near-access query problem on the base network. The trusted neighbor query algorithm can solve the problem of neighbor query in the uncertain network environment from the non-deterministic point of view. (4) An uncertain data Top-k query algorithm based on uncertain theory is proposed. the meta-establishment model in the uncertainty data set is a non-deterministic network, and the top-k query of the ordered tuple is equivalent to the uncertainty measure relation of the edge in the corresponding sample graph, and the sample graph is classified according to the sorting position of the included edge, The algorithm avoids the calculation of the uncertainty measure value of all the tuples in the sample graph, and improves the top-k query calculation efficiency of the uncertainty data. In the uncertain data, the top-k query based on the parameterized ranking function is equivalent to a limited query different according to the Top-k value, and the system implementation is completed in combination with the Spark Map-Reduce programming framework.
【學位授予單位】：北京科技大學
【學位級別】：博士
【學位授予年份】：2016
【分類號】：TP309
本文編號：2501680
[Abstract]:In January 2016, the RightScale conducted a survey of the use of public clouds, private clouds and hybrid clouds for more than 1,000 enterprise users worldwide, and the survey found that 95% of the respondents were using the cloud. In the real world, uncertainty is common in various phenomena. In the cloud computing environment, the cloud data in the cloud data center, the migration and the scheduling of the virtual machine and the like have the uncertainty. There are many achievements in the data processing of the uncertainty, and the uncertainty of the entity's data is not enough to cover some of the real problems in the real world. For the non-deterministic processing of the relation between the entities, the existing literature uses the random and fuzzy theory to solve the problem of neighbor query processing. The relationship between the entities is sometimes also subjective, and the subjective uncertainty is neither random nor fuzzy. In reality, many problems can't get the historical data, so we can not use the probability theory to solve the frequency of the event. At this time, it is necessary to evaluate the reliability of the event based on the experience of the experts, which makes the variance of the reliability far greater than the frequency. In order to deal with the subjective uncertainty of cloud data, the process technology of cloud data will be studied with the uncertainty theory. This paper is devoted to the research of the key technology of cloud data query processing and query optimization. Because of the heterogeneity, privacy, privacy protection, incomplete data and inaccurate data, the data of the cloud data center is uncertain, and the relevant research of the uncertainty theory is used for reference and absorption. The cloud data center is abstracted as an uncertainty diagram. According to the path query algorithm of the uncertain graph, the query processing and query optimization of the cloud data are discussed in-depth. The main work and contribution of this paper can be summarized as follows: (1) The cloud data safety protection framework is proposed. The framework mainly includes physical security, virtual network security, cloud operating system security, virtual cluster security, data security, SaaS/ PaaS/ IaaS security, security management and security operation and maintenance level modules. The framework is the same as the traditional security in the aspects of security objective, system resource type and basic security technology, but also has the special security problem, mainly including: the virtualization security problem and some safety problems related to the cloud computing sublease service mode. The framework has better security and protection capabilities in terms of virtualization security, data security, and privacy protection. (2) The risk analysis method for uncertain random fault tree based on cloud data security protection framework is presented. The method is based on the theory of uncertain theory and opportunity, and the fault tree is constructed and analyzed. The fault tree is composed of a logical relationship based on the bottom event. If the failure rate of the bottom event is obtained from the historical data, it is characterized as a random variable: if there is no historical data, it can be obtained from the subjective judgment of the expert and is characterized as an uncertain variable. In addition, the chance of the occurrence of the event is an uncertain random variable, so a hybrid simulation algorithm is constructed to calculate the opportunity for the top event to occur. The proposed cloud data safety protection framework is analyzed by uncertain stochastic fault tree analysis. And (3) the method for querying the trusted neighbor of the network condition is proposed. The method comprises a CMDCD algorithm, a reachable path length calculation (CMDFP) algorithm, a reachable path expectation length calculation (CMDLFP) algorithm, and a conditional trusted k-neighbor query (QMCCK) algorithm. The uncertain network is modeled as an uncertain weight graph, a sample graph, a sample map index, a basic network, a reachable path length and a reachable path expectation length of the uncertain graph are defined, and an efficient and uncertain conditional trusted neighbor query algorithm based on the uncertainty theory is given. The neighbor query on the network is not determined to be equivalently converted into a near-access query problem on the base network. The trusted neighbor query algorithm can solve the problem of neighbor query in the uncertain network environment from the non-deterministic point of view. (4) An uncertain data Top-k query algorithm based on uncertain theory is proposed. the meta-establishment model in the uncertainty data set is a non-deterministic network, and the top-k query of the ordered tuple is equivalent to the uncertainty measure relation of the edge in the corresponding sample graph, and the sample graph is classified according to the sorting position of the included edge, The algorithm avoids the calculation of the uncertainty measure value of all the tuples in the sample graph, and improves the top-k query calculation efficiency of the uncertainty data. In the uncertain data, the top-k query based on the parameterized ranking function is equivalent to a limited query different according to the Top-k value, and the system implementation is completed in combination with the Spark Map-Reduce programming framework.
【學位授予單位】：北京科技大學
【學位級別】：博士
【學位授予年份】：2016
【分類號】：TP309
【相似文獻】
相關期刊論文 前10條
1 ;連續(xù)性數(shù)據(jù)分組怎樣表示最科學?[J];無錫職業(yè)技術學院學報;2012年04期
2 葛楊;徐名海;遲歡;;關于傳輸虛擬化中數(shù)據(jù)分組亂序問題的研究[J];電信科學;2012年10期
3 葉玉杰;邱丘;陳亞軍;張永忠;劉嵩;胡霞敏;;運用Origin軟件處理藥學實驗數(shù)據(jù)[J];藥學實踐雜志;2013年06期
4 董永吉;郭云飛;黃萬偉;黃慧群;;面向深度分組檢測的高速數(shù)據(jù)分組解析結構[J];通信學報;2013年06期
5 高衛(wèi)民;數(shù)據(jù)處理組合軟件的原理與技術[J];計算機應用研究;1991年02期
6 王銳;陳麗;馬方明;;一種電信行業(yè)海量數(shù)據(jù)分組統(tǒng)計方法[J];計算機應用與軟件;2012年12期
7 ;艾法斯推出業(yè)界最真實的LTE服務測試流量場景[J];移動通信;2012年24期
8 李玉海;田苗苗;黃劉生;楊威;;無線傳感網(wǎng)絡中基于數(shù)據(jù)混淆的保護隱私數(shù)據(jù)聚集協(xié)議[J];小型微型計算機系統(tǒng);2013年07期
9 李霽;麻土華;;基于W函數(shù)的數(shù)據(jù)分組方法的算法實現(xiàn)[J];科技通報;2012年05期
10 鄭海鷗,張乃通;無線數(shù)據(jù)擴頻MODEM組網(wǎng)[J];無線電通信技術;2000年03期
相關重要報紙文章 前1條
1 本報記者 趙姍;大數(shù)據(jù)時代來臨，中國準備好了嗎？[N];中國經(jīng)濟時報;2013年
相關博士學位論文 前3條
1 劉海青;大規(guī)模VANET數(shù)據(jù)傳輸策略的研究[D];山東大學;2015年
2 郭長友;基于不確定理論的云數(shù)據(jù)處理關鍵技術研究[D];北京科技大學;2016年
3 劉琴;多用戶共享云計算服務環(huán)境下安全問題研究[D];中南大學;2012年
相關碩士學位論文 前10條
1 張忠賀;Modbus/TCP協(xié)議在WIFI應用通訊下的實現(xiàn)[D];內蒙古大學;2015年
2 喬新生;量子密鑰分發(fā)系統(tǒng)中數(shù)據(jù)協(xié)調算法及軟件設計[D];西安電子科技大學;2014年
3 楊建東;云環(huán)境下網(wǎng)管數(shù)據(jù)查詢系統(tǒng)設計[D];南京郵電大學;2015年
4 汪鵬飛;異構網(wǎng)絡中基于MPTCP多路傳輸?shù)臄?shù)據(jù)調度策略研究[D];華中師范大學;2015年
5 劉娟;橋梁健康監(jiān)測系統(tǒng)的設計與實現(xiàn)[D];電子科技大學;2011年
6 陳娜;基于Hadoop平臺的海量數(shù)據(jù)處理應用[D];吉林大學;2012年
7 蔣杰;基于GPRS的嵌入式系統(tǒng)安全無堵塞通信研究與設計[D];哈爾濱工程大學;2013年
8 蔣穎;面向卷煙制絲線的SPC系統(tǒng)的應用研究與實現(xiàn)[D];湖南大學;2012年
9 羅恩澤;面向大規(guī)模工程機械遠程智能監(jiān)控的無線通信協(xié)議[D];湖南大學;2011年
10 付金光;電力系統(tǒng)不良數(shù)據(jù)辨識的實用軟件開發(fā)[D];鄭州大學;2011年
，本文編號：2501680
本文鏈接：http://www.sikaile.net/shoufeilunwen/xxkjbs/2501680.html
最近更新
教材專著
