發(fā)布時間：2018-12-30 21:11

【摘要】：數(shù)據(jù)是當(dāng)今互聯(lián)網(wǎng)時代最核心最具價值的產(chǎn)物之一,個人在消費(fèi)和使用互聯(lián)網(wǎng)產(chǎn)品的同時產(chǎn)生了大量需要持久化存儲的數(shù)據(jù),個體存儲設(shè)備的解決方式在這樣一個存儲量和存儲要求日益增長的條件下不再能夠滿足個人用戶的要求,因此越來越多的個人用戶成為了云存儲的受眾。云計(jì)算幾十年的發(fā)展使得公有云環(huán)境下的存儲技術(shù)已然十分成熟,商業(yè)環(huán)境下誕生了許多優(yōu)秀的云盤產(chǎn)品,但其核心技術(shù)是對外封閉的,而對于這種面向大量個人用戶的公有云盤系統(tǒng),開源社區(qū)也并沒有完善可靠的整體解決方案。本文基于上述原因?qū)⒐性票P涉及的各個關(guān)鍵模塊和技術(shù)點(diǎn)進(jìn)行逐個研究和設(shè)計(jì),提出了一整套可靠的云盤系統(tǒng)實(shí)現(xiàn)架構(gòu)。首先對云盤的遠(yuǎn)程文件系統(tǒng)操作機(jī)制的原理和技術(shù)進(jìn)行了分析,研究并設(shè)計(jì)出CFSRPC協(xié)議,給出了協(xié)議的客戶端和服務(wù)端的具體格式定義和二者的通信機(jī)制,云盤系統(tǒng)通過對該協(xié)議的實(shí)現(xiàn),可以透明化遠(yuǎn)程文件操作的細(xì)節(jié),方便客戶端和服務(wù)端上層業(yè)務(wù)的實(shí)現(xiàn)。與此同時為該協(xié)議設(shè)計(jì)了事務(wù)機(jī)制以顆�；募䝼鬏敺⻊�(wù),設(shè)計(jì)了CSDA算法在協(xié)議級別動態(tài)調(diào)節(jié)文件傳輸?shù)姆謮K大小,增加了該協(xié)議對云盤系統(tǒng)性能的優(yōu)化。接著對云盤系統(tǒng)的核心業(yè)務(wù)流程進(jìn)行分析,針對云盤系統(tǒng)依賴的多個關(guān)鍵模塊進(jìn)行了分析和整合設(shè)計(jì),總結(jié)出以訪問層、管理層、存儲層為核心的整體軟件架構(gòu)�；贑eph分布式文件系統(tǒng)設(shè)計(jì)核心存儲模塊,研究分析CephFS和MDS的實(shí)現(xiàn)原理并根據(jù)云盤系統(tǒng)架構(gòu)和CFSRPC協(xié)議的定義,結(jié)合多路IO復(fù)用機(jī)制設(shè)計(jì)實(shí)現(xiàn)了Ceph的接入模塊CCD。將云盤系統(tǒng)的請求分化為文件系統(tǒng)操作請求和管理功能操作請求,分別以CCD模塊和管理功能模塊來接受和處理,中間加入LVS+Keepalived的負(fù)載機(jī)制使得CCD節(jié)點(diǎn)和管理功能節(jié)點(diǎn)支持橫向擴(kuò)展。參考Keystone的令牌認(rèn)證機(jī)制設(shè)計(jì)實(shí)現(xiàn)了獨(dú)立認(rèn)證系統(tǒng)。以NW.js為框架根據(jù)CFSRPC協(xié)議的定義設(shè)計(jì)實(shí)現(xiàn)了客戶端。最后將整套云盤系統(tǒng)在測試環(huán)境下進(jìn)行部署,完成了認(rèn)證流程和云盤基本操作的測試,并對單點(diǎn)CCD進(jìn)行了性能測試,確認(rèn)整體系統(tǒng)能夠達(dá)到公有云生產(chǎn)環(huán)境的性能要求。
[Abstract]:Data is one of the core and most valuable products in the Internet age. While consuming and using Internet products, individuals have produced a large number of data that need to be persisted and stored at the same time. The solution of individual storage devices can no longer meet the requirements of individual users under the condition of increasing storage capacity and storage requirements, so more and more individual users become the audience of cloud storage. With the development of cloud computing for decades, the storage technology in the public cloud environment has been very mature, and many excellent cloud disk products have been born in the commercial environment, but its core technology is closed to the outside. For this public cloud disk system for a large number of individual users, the open source community does not have a sound and reliable overall solution. Based on the above reasons, the key modules and technical points involved in the public cloud disk are studied and designed one by one, and a set of reliable cloud disk system implementation architecture is proposed. Firstly, the principle and technology of the remote file system operation mechanism of cloud disk are analyzed, and the CFSRPC protocol is studied and designed. The specific format definition of client and server and the communication mechanism of the two protocols are given. Through the implementation of the protocol, the cloud disk system can make the details of the remote file operation transparent and facilitate the implementation of the upper layer business of the client and server. At the same time, the transaction mechanism is designed for the protocol to granulate file transfer service, and the CSDA algorithm is designed to dynamically adjust the block size of file transfer at the protocol level, which increases the performance optimization of the cloud disk system. Then the core business process of cloud disk system is analyzed and integrated design for several key modules of cloud disk system is analyzed and integrated. The overall software architecture with access layer management layer and storage layer as the core is summarized. Based on the design of core storage module of Ceph distributed file system, the realization principle of CephFS and MDS is studied and analyzed. According to the definition of cloud disk system architecture and CFSRPC protocol, the CCD. access module of Ceph is designed and implemented with the multichannel IO multiplexing mechanism. The request of cloud disk system is divided into file system operation request and management function operation request, which are received and processed by CCD module and management function module respectively. The load mechanism of adding LVS Keepalived in the middle enables the CCD node and the management function node to support horizontal expansion. An independent authentication system is designed and implemented with reference to the token authentication mechanism of Keystone. According to the definition of CFSRPC protocol, the client is designed and implemented based on NW.js. Finally, the whole cloud disk system is deployed in the test environment, the authentication process and the basic operation of the cloud disk are tested, and the performance of the single point CCD is tested to confirm that the whole system can meet the performance requirements of the public cloud production environment.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前10條

1 穆彥良;徐振明;;Ceph存儲中基于溫度因子的CRUSH算法改進(jìn)[J];成都信息工程學(xué)院學(xué)報;2015年06期

2 王文睿;;node-webkit:HTML5桌面應(yīng)用運(yùn)行環(huán)境[J];程序員;2014年01期

3 吉慧;;云存儲產(chǎn)品性能測試方法研究[J];電信技術(shù);2013年07期

4 任宇寧;;云計(jì)算時代的存儲技術(shù)——云存儲[J];科技傳播;2012年03期

5 李楊;李曙東;黃亮;;Samba服務(wù)器的研究與應(yīng)用[J];中國水運(yùn)(下半月);2011年06期

6 吳一民;劉偉安;;基于Fuse的用戶態(tài)文件系統(tǒng)的設(shè)計(jì)[J];微計(jì)算機(jī)信息;2010年06期

7 王曰芬;章成志;張蓓蓓;吳婷婷;;數(shù)據(jù)清洗研究綜述[J];現(xiàn)代圖書情報技術(shù);2007年12期

8 崔濱;萬旺根;余小清;樓順天;;基于EPOLL機(jī)制的LINUX網(wǎng)絡(luò)游戲服務(wù)器實(shí)現(xiàn)方法[J];微計(jì)算機(jī)信息;2006年21期

9 李春林,王麗芳,蔣澤軍,陳小平;基于身份認(rèn)證技術(shù)安全體系的研究[J];微電子學(xué)與計(jì)算機(jī);2005年04期

10 史芳麗,周亞莉;Linux系統(tǒng)中虛擬文件系統(tǒng)內(nèi)核機(jī)制研究[J];陜西師范大學(xué)學(xué)報(自然科學(xué)版);2005年01期

相關(guān)會議論文 前1條

1 張畢濤;辛陽;;基于Ceph的海量小文件存儲的優(yōu)化方法[A];第十屆中國通信學(xué)會學(xué)術(shù)年會論文集[C];2014年

，

本文編號：2396151