發(fā)布時間：2018-03-16 03:10

  本文選題：元數(shù)據(jù)　切入點：元數(shù)據(jù)組織　出處：《華中科技大學》2016年碩士論文　論文類型：學位論文

【摘要】：隨著大數(shù)據(jù)時代的到來,面向大數(shù)據(jù)的存儲系統(tǒng)紛紛出現(xiàn)。不斷增長的數(shù)據(jù)量,使得集中式元數(shù)據(jù)管理系統(tǒng)的負擔越來越重,逐漸成為大數(shù)據(jù)存儲的瓶頸。為此,人們提出了多種分布式元數(shù)據(jù)管理方法,但由于元數(shù)據(jù)的結(jié)構(gòu)類型復雜多樣,目前尚沒有一種方法能夠同時改善元數(shù)據(jù)管理的性能和擴展性。提出了一種文件路徑和屬性信息分離的分布式元數(shù)據(jù)組織方法。將元數(shù)據(jù)組織成目錄索引和元數(shù)據(jù)屬性信息兩個部分,通過構(gòu)建目錄索引,將元數(shù)據(jù)以目錄或小于目錄為單位劃分到不同的桶(Bucket)內(nèi),再根據(jù)元數(shù)據(jù)服務器集群的負載情況將桶指派到不同的元數(shù)據(jù)服務器上。方法利用目錄索引和桶提高元數(shù)據(jù)的管理性能;通過構(gòu)建目錄索引時考慮集群負載情況,實現(xiàn)元數(shù)據(jù)管理的可擴展性。此外,提出基于該方法的元數(shù)據(jù)位置緩存策略,策略解決了位置緩存信息不一致的問題,縮短了元數(shù)據(jù)管理的流程。測試結(jié)果表明,提出的方法能獲得較高的管理性能,特別適合高并發(fā)的情況;具有良好的可擴展性和較好的訪問局部性,而且可以不限制目錄的大小;避免了重命名元數(shù)據(jù)造成的不必要的遷移。與集中式元數(shù)據(jù)管理方法對比,方法采用單一元數(shù)據(jù)服務器時,元數(shù)據(jù)的創(chuàng)建、查詢等操作性能都有了數(shù)倍的提升。
[Abstract]:With the arrival of big data's era, the storage system for big data appeared one after another. The increasing amount of data makes the burden of centralized metadata management system become more and more heavy, and gradually becomes the bottleneck of big data storage. A variety of distributed metadata management methods have been proposed, but because of the complexity and diversity of the structure of metadata, At present, there is no method to improve the performance and scalability of metadata management simultaneously. A distributed metadata organization method, which separates file path and attribute information, is proposed. The metadata is organized into directory index and metadata. According to two parts of attribute information, By building a directory index, the metadata is divided into different buckets in directories or smaller than directories. Then according to the load of metadata server cluster, the buckets are assigned to different metadata servers. Methods Directory index and bucket are used to improve the management performance of metadata. In addition, a metadata location caching strategy based on this method is proposed, which solves the problem of inconsistent location cache information and shortens the process of metadata management. The test results show that, The proposed method can achieve high management performance, especially suitable for high concurrency, have good scalability and good access locality, and can not limit the size of the directory. Compared with centralized metadata management method, when using single metadata server, the operation performance of metadata creation and query has been improved several times.
【學位授予單位】：華中科技大學
【學位級別】：碩士
【學位授予年份】：2016
【分類號】：TP311.13

【參考文獻】

相關(guān)期刊論文 前7條

1 肖中正;陳寧江;魏峻;張文博;;一種面向海量存儲系統(tǒng)的高效元數(shù)據(jù)集群管理方案[J];計算機研究與發(fā)展;2015年04期

2 羅軍;陳席林;李文生;;高效Key-Value持久化緩存系統(tǒng)的實現(xiàn)[J];計算機工程;2014年03期

3 周江;王偉平;孟丹;馬燦;古曉艷;蔣杰;;面向大數(shù)據(jù)分析的分布式文件系統(tǒng)關(guān)鍵技術(shù)[J];計算機研究與發(fā)展;2014年02期

4 徐鵬;陳思;蘇森;;互聯(lián)網(wǎng)應用PaaS平臺體系結(jié)構(gòu)[J];北京郵電大學學報;2012年01期

5 韓君易;;NoSQL數(shù)據(jù)庫解決方案Tair淺析[J];電子商務;2011年09期

6 馮幼樂;朱六璋;;CEPH動態(tài)元數(shù)據(jù)管理方法分析與改進[J];電子技術(shù);2010年09期

7 羅達強;;探析Windows Azure Platform微軟云計算平臺[J];硅谷;2010年16期

，

本文編號：1618057