發(fā)布時間：2018-08-20 13:29

【摘要】：隨著信息技術(shù)的飛速發(fā)展,特別是移動互聯(lián)網(wǎng)、物聯(lián)網(wǎng)等的發(fā)展,數(shù)據(jù)呈現(xiàn)出了爆發(fā)式增長,我們已經(jīng)步入了海量數(shù)據(jù)的時代。傳統(tǒng)的存儲管理方式已經(jīng)不再滿足當(dāng)前的存儲現(xiàn)狀,如何有效的存儲、管理、維護這些數(shù)據(jù)已經(jīng)成為了一個熱點問題。 云存儲技術(shù)的飛速發(fā)展,使得云存儲成為了一種新型的數(shù)據(jù)存儲解決方案。越來越多的開發(fā)者和企業(yè)將數(shù)據(jù)遷移到云端平臺上,以降低數(shù)據(jù)管理和運維成本,并減輕海量數(shù)據(jù)的沖擊,但是云存儲目前還處于發(fā)展階段,各種技術(shù)和相應(yīng)的法律法規(guī)并不成熟和完善,因此存儲在云端的數(shù)據(jù)并不是萬無一失的,很有可能因為一些突發(fā)事件導(dǎo)致用戶數(shù)據(jù)的丟失,或者機密數(shù)據(jù)的信息泄露,考慮到這些因素企業(yè)內(nèi)部比較敏感重要的數(shù)據(jù)是不適合存放在現(xiàn)有的商用云存儲系統(tǒng)之上的。 本文在綜合分析了目前國內(nèi)外云存儲技術(shù)的發(fā)展現(xiàn)狀,借鑒了目前最穩(wěn)定、最成熟的云存儲產(chǎn)品Amazon S3中的技術(shù)方案,考慮到企業(yè)內(nèi)部現(xiàn)有的硬件存儲設(shè)備,提出了一個具有高可擴展性、高可靠性、兼容不同存儲設(shè)備的分布式云存儲解決方案——基于HDFS的分布式云存儲系統(tǒng)。該系統(tǒng)分為三個部分：底層數(shù)據(jù)存儲部分、中間邏輯處理部分、前端訪問部分,整個系統(tǒng)是構(gòu)建在分布式文件系統(tǒng)HDFS之上的,充分利用了其在數(shù)據(jù)災(zāi)備、容錯糾錯、數(shù)據(jù)恢復(fù)方面的優(yōu)秀表現(xiàn),在底層存儲系統(tǒng)之上設(shè)計實現(xiàn)了文件讀寫模塊,在兼容Amazon S3協(xié)議的基礎(chǔ)上,設(shè)計實現(xiàn)了面向前端請求的代理模塊、核心業(yè)務(wù)邏輯處理模塊、基于數(shù)據(jù)庫的元數(shù)據(jù)存儲模塊,提供了兩種服務(wù)訪問方式：Web前端瀏覽器訪問、SDK訪問方式,為了確保數(shù)據(jù)請求在傳輸過程中的安全性和完整性,設(shè)計實現(xiàn)了安全控制模塊,這樣就構(gòu)建了一個具有高可擴展、高容錯、可靠、安全的分布式云存儲系統(tǒng),最后本文完成了整個云存儲系統(tǒng)的分布式部署和測試。
[Abstract]:With the rapid development of information technology, especially the development of mobile Internet, Internet of things and so on, the data show explosive growth, we have entered the era of massive data. The traditional storage management method is no longer satisfied with the current storage situation. How to effectively store, manage and maintain these data has become a hot issue. With the rapid development of cloud storage technology, cloud storage has become a new data storage solution. More and more developers and enterprises migrate data to cloud platform to reduce the cost of data management and operation and reduce the impact of massive data, but cloud storage is still in the development stage. Various technologies and corresponding laws and regulations are not mature and perfect, so the data stored in the cloud is not foolproof. It is very likely that some unexpected events will lead to the loss of user data or the disclosure of confidential information. Considering these factors, the more sensitive and important data within the enterprise is not suitable to store on the existing commercial cloud storage system. In this paper, the development status of cloud storage technology at home and abroad is comprehensively analyzed, and the most stable and mature cloud storage product Amazon S3 is used for reference, taking into account the existing hardware storage devices within the enterprise. A distributed cloud storage solution with high scalability, high reliability and compatibility with different storage devices is proposed, which is a distributed cloud storage system based on HDFS. The system is divided into three parts: the underlying data storage part, the middle logic processing part, the front-end access part. The whole system is built on the distributed file system HDFS, which makes full use of the system in data disaster preparedness, fault tolerance and error correction. On the basis of compatible Amazon S3 protocol, the paper designs and implements the proxy module for front-end request and the core business logic processing module. The metadata storage module based on database provides two service access modes: Web front-end browser access SDK access mode. In order to ensure the security and integrity of data request in the process of transmission, the security control module is designed and implemented. In this way, a distributed cloud storage system with high extensibility, high fault tolerance, reliability and security is constructed. Finally, the distributed deployment and test of the whole cloud storage system are completed.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2013
【分類號】：TP333

【參考文獻】

相關(guān)期刊論文 前10條

1 何海釗;王虎奇;;數(shù)據(jù)庫連接與訪問技術(shù)的研究與應(yīng)用[J];電腦知識與技術(shù);2009年01期

2 蔡鍵;王樹梅;;基于Google的云計算實例分析[J];電腦知識與技術(shù);2009年25期

3 郭鋒;劉建偉;;基于Socket的局域網(wǎng)絡(luò)通信軟件開發(fā)[J];電子科技;2009年05期

4 李向軍;;基于云計算的數(shù)據(jù)存儲系統(tǒng)研究[J];硅谷;2010年19期

5 張曉明;姜本臣;;一種AJAX結(jié)合CGI的嵌入式瘦Web服務(wù)器的研究[J];信息技術(shù);2011年12期

6 董勇,周恩強,陳娟;基于Infiniband技術(shù)構(gòu)建高性能分布式文件系統(tǒng)-Lustre[J];計算機工程與應(yīng)用;2005年22期

7 曾賽峰;朱立谷;李強;張福;;企業(yè)級私有云中的虛擬化實現(xiàn)[J];計算機工程與應(yīng)用;2010年36期

8 祝志敏;修磊;彭新光;;NFS網(wǎng)絡(luò)文件系統(tǒng)技術(shù)在銀行批量收付交易中的應(yīng)用[J];科技情報開發(fā)與經(jīng)濟;2007年17期

9 吳大勝;;基于超文本標(biāo)記語言的信息隱藏方法研究與實現(xiàn)[J];軟件導(dǎo)刊;2011年10期

10 馬俊;楊樹軍;;高性能集群文件系統(tǒng)的研究[J];計算機工程與設(shè)計;2006年13期

相關(guān)碩士學(xué)位論文 前2條

1 呂寧;基于REST架構(gòu)的Atom Feed存儲集群研究與設(shè)計[D];北京交通大學(xué);2008年

2 孫楊;基于REST風(fēng)格構(gòu)建Web服務(wù)的研究與應(yīng)用[D];電子科技大學(xué);2009年

，

本文編號：2193784