【摘要】：隨著信息技術(shù)的不斷發(fā)展，數(shù)據(jù)日益成為人們?nèi)粘Ｉ钪兄匾Y源。據(jù)統(tǒng)計，2010年全球創(chuàng)建、存儲和復(fù)制的數(shù)字信息總量已達(dá)到1.2ZB，2011年的數(shù)據(jù)量達(dá)到里程碑式的1.8ZB，而且這種增長還在加速，預(yù)計2015年將達(dá)到近8ZB。爆炸式增長的數(shù)據(jù)必然帶來存儲設(shè)備的持續(xù)增加。目前，海量數(shù)據(jù)存儲環(huán)境下的現(xiàn)代數(shù)據(jù)中心的存儲節(jié)點(diǎn)規(guī)模少則幾萬多則幾十萬，但在規(guī)模巨大的存儲環(huán)境系統(tǒng)中，磁盤損毀或者存儲節(jié)點(diǎn)失效已成為一種常態(tài)行為；與此同時，因網(wǎng)絡(luò)連接設(shè)備或者存儲節(jié)點(diǎn)其它元器件造成的數(shù)據(jù)不可訪問或者丟失現(xiàn)象也時有發(fā)生。為了滿足日益擴(kuò)展的數(shù)據(jù)存儲需求，人們對數(shù)據(jù)存儲的可靠性，可用性等相關(guān)特性提出了更高的要求，傳統(tǒng)的常規(guī)技術(shù)手段根本無法應(yīng)對當(dāng)前的形式，如何實(shí)現(xiàn)海量數(shù)據(jù)的低冗余度高可靠性存儲已經(jīng)成為業(yè)界面臨的一個巨大挑戰(zhàn)。 因而，本文針對構(gòu)建低冗余度高可靠性海量數(shù)據(jù)存儲系統(tǒng)的關(guān)鍵問題，在總結(jié)了當(dāng)今數(shù)據(jù)可靠性增強(qiáng)理論和海量數(shù)據(jù)存儲系統(tǒng)基本架構(gòu)的基礎(chǔ)上，對高性能數(shù)據(jù)容刪數(shù)據(jù)布局算法以及高可靠性存儲架構(gòu)等方面進(jìn)行了深入的研究，取得了一定的進(jìn)展，具體如下： 1．針對已在數(shù)據(jù)存儲系統(tǒng)中有廣泛應(yīng)用的RAID技術(shù)，提出了一種新的基于異或運(yùn)算的水平陣列糾刪碼：EX-ENOD碼。該碼能夠容許任意三列的隨機(jī)刪除錯，并具有極大距離可分性質(zhì)。根據(jù)該碼的幾何構(gòu)造特點(diǎn)，提出了一種具有低計算復(fù)雜度的譯碼方法，該方法的計算復(fù)雜度要低于目前已知的其它可糾三列隨機(jī)刪除錯糾刪碼的譯碼方法。同時，該編碼方法具有明顯的通用性，可以擴(kuò)展應(yīng)用到STAR碼、EEOD碼的譯碼過程。 2．針對海量存儲系統(tǒng)規(guī)模不斷擴(kuò)大，，可靠性要求不斷提高的需求，本文將在{0,1}符號域上的范德蒙系統(tǒng)編碼方法引入存儲系統(tǒng)中來。該種編碼方法繼承了傳統(tǒng)有限域上構(gòu)建的范德蒙編碼的參數(shù)不受存儲節(jié)點(diǎn)規(guī)模、容錯參數(shù)限制的優(yōu)良特性，而且存儲效率達(dá)到最高，同時該方法突破了傳統(tǒng)有限域上構(gòu)建的編碼需要大量查詢運(yùn)算的弊端。基于該編碼方法構(gòu)建的存儲系統(tǒng)，在保證其數(shù)據(jù)仍然可用的情況下，最多可容許系統(tǒng)內(nèi)部一半的存儲節(jié)點(diǎn)發(fā)生損毀，在該種情況下，系統(tǒng)僅需要與原數(shù)據(jù)相同的冗余數(shù)據(jù)量。 3．針對{0,1}符號域上編碼矩陣的特點(diǎn)，根據(jù)編碼矩陣中各行向量中“1”元素的分布，提出了可降低編譯碼計算復(fù)雜度的優(yōu)化算法。文章同時針對傳統(tǒng)譯碼重構(gòu)過程重構(gòu)帶寬較高的缺點(diǎn)，提出了基于校驗(yàn)矩陣的譯碼方法，并根據(jù)校驗(yàn)矩陣列向量的特點(diǎn)和存儲系統(tǒng)所需要重構(gòu)數(shù)據(jù)的數(shù)目，給出了一種低帶寬重構(gòu)算法。該種低帶寬重構(gòu)算法，可以推廣到所有構(gòu)建在{0,1}符號域上的編碼存儲系統(tǒng)。 4．根據(jù)編碼冗余策略數(shù)據(jù)布局的特點(diǎn)，設(shè)計了一種低冗余度高可靠性海量數(shù)據(jù)存儲系統(tǒng)基礎(chǔ)架構(gòu)。系統(tǒng)將數(shù)據(jù)消冗和編碼冗余可靠性增強(qiáng)技術(shù)納入統(tǒng)一的基礎(chǔ)架構(gòu)，并針對編碼冗余數(shù)據(jù)的分布特點(diǎn)，進(jìn)行了存儲節(jié)點(diǎn)的節(jié)能設(shè)計；針對數(shù)據(jù)使用特點(diǎn)提出了非均等存儲及自適應(yīng)讀取策略；并提出了數(shù)據(jù)消冗與數(shù)據(jù)驗(yàn)證協(xié)同進(jìn)行的運(yùn)行策略。
[Abstract]:With the continuous development of information technology, data is increasingly becoming an important resource in people's daily life. According to statistics, the total amount of digital information created in 2010, stored and copied has reached 1.2ZB, the amount of data in 2011 reached a milestone of 1.8ZB, and this growth is accelerating, it is expected to reach nearly 8ZB. explosive growth in 2015 data. At present, the scale of storage nodes in modern data centers under mass data storage environment is tens of thousands or hundreds of thousands, but disk damage or storage node failure has become a normal behavior in large-scale storage environment systems; at the same time, because of network connection equipment or storage. In order to meet the ever-expanding demand for data storage, people put forward higher requirements for the reliability and availability of data storage. Traditional conventional technical means can not cope with the current form, how to achieve massive data. Low redundancy and high reliability storage has become a huge challenge for the industry.
Therefore, aiming at the key problems of constructing low redundancy and high reliability mass data storage system, this paper summarizes the theory of data reliability enhancement and the basic architecture of mass data storage system, and makes a thorough study on high performance data deletion tolerance data layout algorithm and high reliability storage architecture. Some progress has been made as follows:
1. A new horizontal array erasure code, EX-ENOD code, is proposed for RAID technology which has been widely used in data storage systems. The code can allow random deletion of arbitrary three columns and has the property of maximum distance separability. The computational complexity of the proposed method is lower than that of other known decoding methods for three-column random deletion and erasure codes.
2. In order to meet the requirement of increasing scale and reliability of mass storage system, this paper introduces Vandermond system coding method in {0,1} symbol field into storage system. The storage system based on this coding method can allow up to half of the storage nodes in the system to be damaged under the condition that the data is still available. The system only needs the same amount of redundant data as the original data.
3. According to the characteristics of the encoding matrix over {0,1} symbol field and the distribution of `1'elements in each vector of the encoding matrix, an optimization algorithm is proposed to reduce the computational complexity of encoding and decoding. The characteristics of array vectors and the number of data to be reconstructed by the storage system are described. A low bandwidth reconstructing algorithm is proposed. The algorithm can be extended to all coded storage systems built on {0,1} symbol domains.
4. According to the characteristics of data layout of coding redundancy strategy, a low-redundancy and high-reliability mass data storage system infrastructure is designed. The strategy of non-uniform storage and self-adaptive reading is proposed according to the characteristics of data usage.
【學(xué)位授予單位】：電子科技大學(xué)
【學(xué)位級別】：博士
【學(xué)位授予年份】：2013
【分類號】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前6條

1 萬武南;索望;張金全;陳運(yùn);;基于EEOD碼的一種有效的數(shù)據(jù)分布策略[J];電子科技大學(xué)學(xué)報;2007年05期

2 董歡慶;李戰(zhàn)懷;林偉;;RAID-VCR:一種能夠承受三個磁盤故障的RAID結(jié)構(gòu)[J];計算機(jī)學(xué)報;2006年05期

3 萬武南;吳震;陳運(yùn);王曉京;;一種基于3容錯陣列碼的RAID數(shù)據(jù)布局[J];計算機(jī)學(xué)報;2007年10期

4 常乾;許胤龍;項利萍;潘玉彪;;基于EVENODD碼的單盤故障快速恢復(fù)算法[J];計算機(jī)應(yīng)用與軟件;2011年06期

5 敖莉;舒繼武;李明強(qiáng);;重復(fù)數(shù)據(jù)刪除技術(shù)[J];軟件學(xué)報;2010年05期

6 王意潔;孫偉東;周松;裴曉強(qiáng);李小勇;;云計算環(huán)境下的分布存儲關(guān)鍵技術(shù)[J];軟件學(xué)報;2012年04期

相關(guān)博士學(xué)位論文 前4條

1 魏青松;大規(guī)模分布式存儲技術(shù)研究[D];電子科技大學(xué);2004年

2 周旭;面向Internet的大規(guī)模分布式存儲技術(shù)研究[D];電子科技大學(xué);2004年

3 萬武南;分布式安全存儲系統(tǒng)糾刪碼技術(shù)的研究[D];中國科學(xué)院研究生院（成都計算機(jī)應(yīng)用研究所）;2006年

4 吳素貞;磁盤陣列高可用技術(shù)研究[D];華中科技大學(xué);2010年

本文編號：2214325