發(fā)布時間：2018-08-16 15:15

【摘要】：隨著信息時代的發(fā)展，海量數(shù)據(jù)的存儲傳輸成為目前必須解決的一個重要問題。云存儲為海量數(shù)據(jù)的存儲提供了很好的解決方案，但是缺乏標準化的云存儲應用程序編程接口，極大的限制了云存儲的應用。 缺乏標準化的云存儲應用程序編程接口使云網(wǎng)關成為云存儲必不可少的重要組成部分，云網(wǎng)關已經(jīng)成為應用和云提供商應用程序編程接口之間協(xié)議的翻譯，但是由于云網(wǎng)關很少支持企業(yè)級的其它服務，限制了它的應用范圍，目前大部分被用于歸檔和備份。云網(wǎng)關的問題在于它增加了云存儲的復雜性及限制了云存儲的性能，所以從性能及簡潔性角度來看，它并不適合作為理想的主要應用。 針對備份歸檔文件中的高冗余問題，云網(wǎng)關上設計實現(xiàn)重復數(shù)據(jù)刪除功能，能為云網(wǎng)關與云平臺間的通信減少網(wǎng)絡帶寬同時減小數(shù)據(jù)在云平臺上的存儲容量。云網(wǎng)關上的重復數(shù)據(jù)刪除功能的設計與實現(xiàn)主要包括在云網(wǎng)關上對文件分塊，將分塊文件計算數(shù)據(jù)指紋，，將數(shù)據(jù)指紋與哈希表中指紋進行比對，丟棄重復數(shù)據(jù)，將文件信息和非重復的數(shù)據(jù)分離獨立保存在Swift云平臺上。 通過與不進行重復數(shù)據(jù)刪除的云網(wǎng)關性能對比表明，在云網(wǎng)關上進行重復數(shù)據(jù)刪除對于高冗余數(shù)據(jù)能夠減少76%~91%的重復數(shù)據(jù)存儲容量，節(jié)省了70%~86%的網(wǎng)絡帶寬，提高了云網(wǎng)關的性能，且對云網(wǎng)關的響應開銷影響較小。
[Abstract]:With the development of the information age, the storage and transmission of massive data has become an important problem that must be solved. Cloud storage provides a good solution for mass data storage, but the lack of standardized cloud storage application programming interface greatly limits the application of cloud storage. The lack of standardized cloud storage application programming interface makes cloud gateway an essential part of cloud storage, and cloud gateway has become the translation of protocol between application and cloud provider application programming interface. However, because cloud gateways rarely support other enterprise-level services, which limits its application, most of them are currently used for archiving and backup. The problem of cloud gateway is that it increases the complexity of cloud storage and limits the performance of cloud storage, so it is not suitable for the ideal main application from the point of view of performance and simplicity. Aiming at the problem of high redundancy in backup archival files, the repeated data deletion function is designed and implemented on cloud gateway, which can reduce the network bandwidth and the storage capacity of data on cloud platform for the communication between cloud gateway and cloud platform. The design and implementation of duplicate data deletion function on cloud gateway mainly include dividing files into blocks, calculating data fingerprints, comparing data fingerprints with fingerprints in hash table, and discarding duplicate data. Separate file information from non-duplicate data and store it independently on the Swift cloud platform. Compared with the cloud gateway without repeated data deletion, it is shown that repeated data deletion on cloud gateway can reduce the storage capacity of 76 / 91% of repetitive data and save 70 / 86% of network bandwidth for high redundant data. The performance of the cloud gateway is improved, and the response overhead of the cloud gateway is less affected.
【學位授予單位】：華中科技大學
【學位級別】：碩士
【學位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關博士學位論文 前4條

1 楊天明;網(wǎng)絡備份中重復數(shù)據(jù)刪除技術研究[D];華中科技大學;2010年

2 黃莉;基于語義關聯(lián)的重復數(shù)據(jù)清理技術研究[D];華中科技大學;2011年

3 譚玉娟;數(shù)據(jù)備份系統(tǒng)中數(shù)據(jù)去重技術研究[D];華中科技大學;2012年

4 王燦;基于在線重復數(shù)據(jù)消除的海量數(shù)據(jù)處理關鍵技術研究[D];電子科技大學;2012年

本文編號：2186369