【摘要】：近年來公共安全威脅愈發(fā)嚴峻,將安防監(jiān)控技術(shù)應(yīng)用到公共安全中刻不容緩�，F(xiàn)有安防監(jiān)控領(lǐng)域?qū)?shù)據(jù)的存儲普遍采用普通分布式存儲系統(tǒng)架構(gòu)。然而,這樣的存儲架構(gòu)下,系統(tǒng)使用多副本策略保證數(shù)據(jù)的可靠性,存儲空間利用率低;此外,多數(shù)系統(tǒng)使用輪詢調(diào)度算法調(diào)度存儲節(jié)點,忽略了存儲節(jié)點的負載均衡性問題,導致節(jié)點性能抖動較大;而且,在數(shù)據(jù)丟失恢復任務(wù)中,沒有考慮視頻流存儲的有效時長等因素,數(shù)據(jù)恢復價值較低。本系統(tǒng)將分布式存儲架構(gòu)技術(shù)與安防監(jiān)控行業(yè)相結(jié)合,設(shè)計與實現(xiàn)了元數(shù)據(jù)服務(wù)器,旨在提高監(jiān)控數(shù)據(jù)的可靠性和系統(tǒng)可擴展性,在接入互聯(lián)網(wǎng)的情況下,支持用戶在合法權(quán)限內(nèi),獲取任意攝像頭的任意時間的數(shù)據(jù)。元數(shù)據(jù)服務(wù)器結(jié)合時間戳策略,設(shè)計了基于糾刪碼的分布式存儲系統(tǒng)和卷式元數(shù)據(jù)管理結(jié)構(gòu),實現(xiàn)了對存儲任務(wù)和恢復任務(wù)的均衡調(diào)度。具體研究內(nèi)容如下:1.設(shè)計和實現(xiàn)了時間戳策略來維護存儲系統(tǒng)中不同服務(wù)器之間卷信息元數(shù)據(jù)的一致性。存儲系統(tǒng)中的每個服務(wù)器同時記錄系統(tǒng)中所有卷數(shù)據(jù)的時間戳信息,以元數(shù)據(jù)服務(wù)器上的卷時間戳信息為系統(tǒng)唯一參考標準,判斷其余模塊中卷信息元數(shù)據(jù)的有效性。2.設(shè)計和實現(xiàn)了以節(jié)點和卷為索引的兩個維度的元數(shù)據(jù)管理結(jié)構(gòu)。元數(shù)據(jù)服務(wù)器維護每個存儲節(jié)點的系統(tǒng)健康狀態(tài)信息,為任務(wù)的均衡調(diào)度提供參考;同時,記錄每個存儲節(jié)點上數(shù)據(jù)分布情況和卷式數(shù)據(jù)的糾刪碼策略相關(guān)信息,為數(shù)據(jù)丟失后的恢復任務(wù)提供依據(jù)。同時,支持系統(tǒng)節(jié)點的動態(tài)擴展。3.設(shè)計和實現(xiàn)了基于多目標優(yōu)化算法的任務(wù)調(diào)度策略。針對存儲任務(wù)的分發(fā),目標函數(shù)不僅保證節(jié)點的負載均衡,而且讓不同節(jié)點之間的執(zhí)行任務(wù)數(shù)量差值最小;針對恢復任務(wù)優(yōu)先級的制定,目標函數(shù)優(yōu)先選擇數(shù)據(jù)冗余率低、丟失率高、視頻剩余有效時間長的卷,提高系統(tǒng)的可靠性。通過對分布式視頻流存儲系統(tǒng)進行功能和性能測試,元數(shù)據(jù)服務(wù)器達到了對視頻流數(shù)據(jù)存儲、讀取和恢復等功能的控制管理需求;另外,存儲任務(wù)和恢復任務(wù)的調(diào)度策略,不僅降低了存儲節(jié)點的負載失衡機率,而且延遲了低效恢復任務(wù)的執(zhí)行,提高了系統(tǒng)恢復數(shù)據(jù)的后期使用價值。
[Abstract]:In recent years, the threat of public security is becoming more and more serious, so it is urgent to apply security monitoring technology to public safety. General distributed storage system architecture is widely used in data storage in the field of security monitoring and control. However, in this storage architecture, the system uses multi-replica strategy to ensure the reliability of data, and the utilization of storage space is low. In addition, most systems use polling scheduling algorithm to schedule storage nodes, ignoring the load balancing problem of storage nodes, resulting in greater performance jitter. Moreover, in the task of data loss recovery, the effective time of video stream storage is not taken into account, so the value of data recovery is low. This system combines distributed storage architecture technology with security monitoring industry, and designs and implements metadata server, which aims to improve the reliability of monitoring data and system scalability, in the case of access to the Internet, Allows users to access arbitrary camera data at any time within legitimate permissions. The distributed storage system based on erasure code and the volume metadata management structure are designed in the metadata server combined with the timestamp strategy. The balanced scheduling of storage and recovery tasks is realized. The specific research contents are as follows: 1. A timestamp strategy is designed and implemented to maintain the consistency of volume information metadata between different servers in the storage system. Each server in the storage system simultaneously records the timestamp information of all volume data in the system. The volume timestamp information on the metadata server is regarded as the unique reference standard of the system to judge the validity of the volume information metadata in the other modules. 2. A metadata management structure based on nodes and volumes is designed and implemented. The metadata server maintains the system health status information of each storage node and provides a reference for the balanced scheduling of tasks. At the same time, the data distribution on each storage node and the erasure policy of volume data are recorded to provide the basis for the recovery task after data loss. At the same time, support the dynamic expansion of system nodes. 3. The task scheduling strategy based on multi-objective optimization algorithm is designed and implemented. For the distribution of storage tasks, the objective function not only ensures the load balance of nodes, but also minimizes the difference in the number of execution tasks between different nodes. In order to determine the priority of the recovery task, the objective function first selects the volume with low data redundancy rate, high loss rate and long video residual effective time to improve the reliability of the system. By testing the function and performance of the distributed video stream storage system, the metadata server achieves the control and management requirements for the video stream data storage, reading and recovery functions. In addition, the scheduling strategy of storage task and recovery task not only reduces the load imbalance probability of storage node, but also delays the execution of inefficient recovery task, and improves the later application value of system recovery data.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2017
【分類號】：TN948.6;TP333

【參考文獻】

相關(guān)期刊論文 前5條

1 張亞波;蘇藝;;基于Red5服務(wù)器集群負載均衡調(diào)度算法研究[J];微型機與應(yīng)用;2013年21期

2 楊_g劍;林波;;分布式存儲系統(tǒng)中一致性哈希算法的研究[J];電腦知識與技術(shù);2011年22期

3 何佳;;數(shù)據(jù)存儲技術(shù)的比較及發(fā)展趨勢[J];懷化學院學報;2008年11期

4 盧秋波;;視頻監(jiān)控技術(shù)簡介與發(fā)展趨勢[J];安防科技;2007年05期

5 韓德志,謝長生,傅湘林,易法令;一種基于iSCSI的附網(wǎng)存儲服務(wù)器系統(tǒng)的設(shè)計與實現(xiàn)[J];計算機研究與發(fā)展;2004年01期

相關(guān)會議論文 前1條

1 黎燕凌;王磊;;DVR計算機數(shù)字監(jiān)控系統(tǒng)[A];中國工程物理研究院科技年報（2005）[C];2005年

相關(guān)博士學位論文 前1條

1 王平;嵌入式計算機控制系統(tǒng)容錯策略研究[D];中國科學院研究生院（上海微系統(tǒng)與信息技術(shù)研究所）;2004年

相關(guān)碩士學位論文 前5條

1 劉曉東;并行文件系統(tǒng)緩存技術(shù)的研究[D];北京交通大學;2013年

2 黃靖鋒;基于SAN的網(wǎng)絡(luò)存儲技術(shù)研究與應(yīng)用[D];電子科技大學;2010年

3 楊建兵;網(wǎng)絡(luò)存儲安全技術(shù)研究[D];上海交通大學;2007年

4 楊麗娟;網(wǎng)絡(luò)存儲技術(shù)研究及網(wǎng)站存儲系統(tǒng)的設(shè)計與實現(xiàn)[D];西南交通大學;2007年

5 趙恒;星載并行計算機硬件系統(tǒng)可靠性設(shè)計與分析[D];國防科學技術(shù)大學;2005年

，

本文編號：2415832