本文選題：數(shù)據(jù)密集型計(jì)算 + 數(shù)據(jù)網(wǎng)格　； 參考：《福州大學(xué)》2014年碩士論文

【摘要】：互聯(lián)網(wǎng)的高速發(fā)展和網(wǎng)絡(luò)寬帶的普及加快了各行各業(yè)的網(wǎng)絡(luò)化和信息化進(jìn)程,同時(shí)網(wǎng)絡(luò)數(shù)據(jù)規(guī)模的不斷膨脹給計(jì)算機(jī)帶來(lái)了巨大挑戰(zhàn)。海量數(shù)據(jù)的管理能力成為了計(jì)算能力發(fā)展過(guò)程中的性能瓶頸,存儲(chǔ)和處理網(wǎng)絡(luò)數(shù)據(jù)的數(shù)據(jù)處理系統(tǒng)逐漸向數(shù)據(jù)密集型系統(tǒng)轉(zhuǎn)變。在這樣的背景下,數(shù)據(jù)密集型計(jì)算(DIC)應(yīng)運(yùn)而生并引起了廣泛的關(guān)注。數(shù)據(jù)管理問(wèn)題是數(shù)據(jù)密集型計(jì)算系統(tǒng)中的一個(gè)核心問(wèn)題,而副本管理技術(shù)是數(shù)據(jù)管理問(wèn)題中被廣泛采用的一種有效技術(shù)。副本管理包括四個(gè)關(guān)鍵技術(shù)：副本創(chuàng)建、副本選擇、副本替換和副本一致性維護(hù),它在提高數(shù)據(jù)可靠性、均衡網(wǎng)絡(luò)負(fù)載、降低數(shù)據(jù)訪問(wèn)延遲和帶寬消耗方面都能起到很好的效果。在了解數(shù)據(jù)密集型計(jì)算環(huán)境中副本管理技術(shù)的基礎(chǔ)上,本文重點(diǎn)對(duì)副本選擇及替換技術(shù)進(jìn)行了進(jìn)一步的研究。針對(duì)已有策略的不足之處,提出了新的副本管理優(yōu)化技術(shù),主要工作包括如下兩方面：(1)對(duì)于數(shù)據(jù)密集型計(jì)算環(huán)境中的副本選擇,在研究已有策略的基礎(chǔ)上,提出了一種改進(jìn)的基于蟻群算法的副本選擇策略。本文將蟻群算法的無(wú)限正反饋性作為一個(gè)考慮因素,對(duì)副本進(jìn)行概率選擇而不是絕對(duì)選擇,避免了某個(gè)副本被頻繁訪問(wèn)而最終導(dǎo)致網(wǎng)絡(luò)擁塞,進(jìn)而影響正在進(jìn)行的其他數(shù)據(jù)傳輸任務(wù)。然后對(duì)主流的網(wǎng)格仿真器OptorSim進(jìn)行擴(kuò)展,將本文提出的算法在仿真器中實(shí)現(xiàn),并與原算法及仿真器中自帶的副本優(yōu)化算法SimpleOptimiser進(jìn)行仿真對(duì)比實(shí)驗(yàn)。(2)基于最近最久未使用(Least Recently Used, LRU)副本替換策略,提出了LRULR (Least Recently Used and Least Replicas)算法。新策略將整個(gè)數(shù)據(jù)網(wǎng)格的文件分布情況也做為副本替換的考慮因素,能有效提高數(shù)據(jù)密集型計(jì)算中數(shù)據(jù)副本的命中率和訪問(wèn)效率,減少副本復(fù)制次數(shù)及數(shù)據(jù)傳輸帶寬消耗,其主要思想是當(dāng)存儲(chǔ)容量不足時(shí)替換最近最久未使用副本集中全局?jǐn)?shù)量最少的副本。然后在OptorSim中實(shí)現(xiàn)新策略,并將其與LRU算法進(jìn)行對(duì)比試驗(yàn)。本文分別對(duì)數(shù)據(jù)密集型計(jì)算的副本選擇和替換問(wèn)題提出了優(yōu)化策略,并在仿真平臺(tái)上與原策略進(jìn)行對(duì)比實(shí)驗(yàn)。OptorSim上的實(shí)驗(yàn)表明,本文提出的算法在減少平均作業(yè)時(shí)間、降低網(wǎng)絡(luò)帶寬消耗和平衡網(wǎng)絡(luò)負(fù)載方面都具有一定的優(yōu)越性。
[Abstract]:The rapid development of the Internet and the popularization of network broadband accelerate the network and information process of all walks of life. At the same time, the continuous expansion of the network data scale has brought great challenges to the computer. The management ability of mass data has become the performance bottleneck in the process of computing power development, and the data processing system for storage and processing of network data is made. In this context, data intensive computing (DIC) came into being and attracted wide attention. Data management is a core problem in the data intensive computing system, and replica management technology is an effective technique used widely in data management. There are four key technologies: copy creation, copy selection, copy replacement and copy consistency maintenance. It can improve data reliability, balance network load, reduce data access delay and bandwidth consumption. On the basis of replica management technology in data intensive computing environment, this paper focuses on replicas. The selection and replacement technology is further studied. In view of the shortcomings of the existing strategies, a new copy management optimization technology is proposed. The main work includes the following two aspects: (1) the copy selection in the data intensive computing environment, and on the basis of the existing strategies, an improved copy based on ant colony algorithm is proposed. In this paper, in this paper, the infinite positive feedback of ant colony algorithm is considered as a consideration factor, and the copy is chosen instead of absolute choice. It avoids the frequent access of a copy and eventually leads to the network congestion, and then affects the other data transmission tasks being carried out. Then, the mainstream grid emulator OptorSim is extended. The algorithm proposed in this paper is implemented in the emulator, and the simulation contrast experiment with the original algorithm and the copy optimization algorithm SimpleOptimiser in the emulator is simulated. (2) based on the most recent Least Recently Used (LRU) copy replacement strategy, the LRULR (Least Recently Used and Least Replicas) algorithm is proposed. The new strategy will be the whole The file distribution of data grid is also considered as a factor of replica substitution. It can effectively improve the hit rate and access efficiency of data copies in the data intensive computing, reduce the number of replicas and reduce the consumption of data transmission bandwidth. The main idea is to replace the most recent unused copy centralized global number when the storage capacity is insufficient. A small copy. Then a new strategy is implemented in OptorSim and compared with the LRU algorithm. This paper presents an optimization strategy for the copy selection and replacement of data intensive computing, and a comparison experiment on the simulation platform with the original strategy on.OptorSim shows that the algorithm proposed in this paper reduces the average work. Time has advantages in reducing network bandwidth consumption and balancing network load.

【學(xué)位授予單位】：福州大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2014
【分類號(hào)】：TP393.09;TP18

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 鄭湃;崔立真;王海洋;徐猛;;云計(jì)算環(huán)境下面向數(shù)據(jù)密集型應(yīng)用的數(shù)據(jù)布局策略與方法[J];計(jì)算機(jī)學(xué)報(bào);2010年08期

2 宮學(xué)慶;金澈清;王曉玲;張蓉;周傲英;;數(shù)據(jù)密集型科學(xué)與工程:需求和挑戰(zhàn)[J];計(jì)算機(jī)學(xué)報(bào);2012年08期

3 茹加;王曉輝;;數(shù)據(jù)密集型網(wǎng)格示范站點(diǎn)建設(shè)與應(yīng)用取得良好效果[J];中國(guó)科學(xué)院院刊;2012年04期

4 石宣化;金海;;有服務(wù)質(zhì)量保證的數(shù)據(jù)密集型網(wǎng)格應(yīng)用管理研究[J];計(jì)算機(jī)科學(xué);2007年06期

5 王雷;陶偉;;一種數(shù)據(jù)密集型應(yīng)用的數(shù)據(jù)副本管理策略[J];計(jì)算機(jī)系統(tǒng)應(yīng)用;2012年12期

6 陳慶奎,那麗春;一個(gè)面向數(shù)據(jù)密集型網(wǎng)格的通信模型[J];北京航空航天大學(xué)學(xué)報(bào);2004年11期

7 丁杰;奚后瑋;韓海韻;周愛(ài)華;;面向智能電網(wǎng)的數(shù)據(jù)密集型云存儲(chǔ)策略[J];電力系統(tǒng)自動(dòng)化;2012年12期

8 錢鑫;張龍波;田愛(ài)奎;鄧齊志;汪金苗;;一種面向數(shù)據(jù)密集型計(jì)算環(huán)境的聚類算法[J];濟(jì)南大學(xué)學(xué)報(bào)(自然科學(xué)版);2013年01期

9 何頌頌;顧乃杰;任開(kāi)新;;一種面向數(shù)據(jù)密集型應(yīng)用的并行程序執(zhí)行模型[J];小型微型計(jì)算機(jī)系統(tǒng);2013年07期

10 周曉英;;情報(bào)學(xué)進(jìn)展系列論文之七 數(shù)據(jù)密集型科學(xué)研究范式的興起與情報(bào)學(xué)的應(yīng)對(duì)[J];情報(bào)資料工作;2012年02期

相關(guān)會(huì)議論文 前1條

1 鄭湃;崔立真;王海洋;徐猛;;云計(jì)算環(huán)境下面向數(shù)據(jù)密集型應(yīng)用的數(shù)據(jù)布局策略與方法[A];NDBC2010第27屆中國(guó)數(shù)據(jù)庫(kù)學(xué)術(shù)會(huì)議論文集A輯一[C];2010年

相關(guān)重要報(bào)紙文章 前1條

1 劉潤(rùn)生;一種新的范式：數(shù)據(jù)密集型科學(xué)[N];學(xué)習(xí)時(shí)報(bào);2013年

相關(guān)博士學(xué)位論文 前1條

1 馬馮;數(shù)據(jù)密集型計(jì)算環(huán)境下貝葉斯網(wǎng)的學(xué)習(xí)、推理及應(yīng)用[D];云南大學(xué);2013年

相關(guān)碩士學(xué)位論文 前10條

1 張樹(shù)森;聚類分析算法研究及其在數(shù)據(jù)密集型計(jì)算環(huán)境下的實(shí)現(xiàn)[D];山東理工大學(xué);2015年

2 楊玲;數(shù)據(jù)密集型計(jì)算中的副本優(yōu)化策略研究[D];福州大學(xué);2014年

3 石飛燕;數(shù)據(jù)密集型計(jì)算中副本優(yōu)化機(jī)制研究[D];武漢理工大學(xué);2011年

4 鄧齊志;數(shù)據(jù)密集型計(jì)算環(huán)境下數(shù)據(jù)分類算法研究[D];山東理工大學(xué);2013年

5 方啟宇;面向數(shù)據(jù)密集型計(jì)算的貝葉斯網(wǎng)并行學(xué)習(xí)與增量維護(hù)[D];云南大學(xué);2014年

6 張釗寧;數(shù)據(jù)密集型計(jì)算中任務(wù)調(diào)度模型的研究[D];國(guó)防科學(xué)技術(shù)大學(xué);2009年

7 俞棟輝;面向數(shù)據(jù)密集型應(yīng)用的數(shù)據(jù)管理關(guān)鍵技術(shù)研究[D];浙江工業(yè)大學(xué);2013年

8 錢鑫;數(shù)據(jù)密集型計(jì)算環(huán)境下聚類算法的研究[D];山東理工大學(xué);2013年

9 王源;面向數(shù)據(jù)密集型計(jì)算的概率圖模型構(gòu)建方法及實(shí)現(xiàn)[D];云南大學(xué);2013年

10 劉藝琨;基于概率圖模型的數(shù)據(jù)密集型廣告點(diǎn)擊率預(yù)測(cè)系統(tǒng)設(shè)計(jì)與實(shí)現(xiàn)[D];云南大學(xué);2015年

，

本文編號(hào)：1836371