發(fā)布時間：2018-05-24 19:04

  本文選題：集群文件系統(tǒng) + 分布式文件系統(tǒng)��； 參考：《計算機學(xué)報》2017年09期

【摘要】：集群文件系統(tǒng)作為一種典型的分布式文件系統(tǒng)類型,通過集群內(nèi)多個節(jié)點的協(xié)同,消除了單點故障以及性能瓶頸問題,實現(xiàn)了高可用、高性能以及動態(tài)負載均衡,并且具有較高的可擴展性,因此常作為實現(xiàn)和提供云存儲服務(wù)的關(guān)鍵技術(shù)之一.該文針對HDFS集群主要局限于同一數(shù)據(jù)中心內(nèi)部部署且可擴展性受限的問題,提出一種跨數(shù)據(jù)中心集群部署的文件資源調(diào)度機制和金蝶分布式文件服務(wù)KDFS,通過分布式架構(gòu)再設(shè)計,支持多個HDFS集群動態(tài)組網(wǎng)協(xié)同工作;通過引入文件資源池,屏蔽了不同集群之間的文件差異性,能夠面向多應(yīng)用提供透明服務(wù);通過引入彈性存儲與最優(yōu)存儲策略,確保集群資源安全冗余與就近服務(wù)的同時提升了集群的存儲效率.實驗和實踐證明,跨HDFS集群的文件資源調(diào)度機制不但解決了HDFS集群可擴展性受限的問題,同時通過跨數(shù)據(jù)中心部署,實現(xiàn)了集群文件異地冗余災(zāi)備、跨數(shù)據(jù)中心負載均衡以及文件就近存取服務(wù),有效地提高了應(yīng)用使用KDFS存儲服務(wù)的體驗.
[Abstract]:As a typical type of distributed file system, cluster file system eliminates single point fault and performance bottleneck problem, and realizes high availability, high performance and dynamic load balancing through the cooperation of multiple nodes in the cluster. Because of its high scalability, it is often used as one of the key technologies to implement and provide cloud storage services. Aiming at the problem that HDFS cluster is mainly confined to the same data center and its expansibility is limited, this paper proposes a file resource scheduling mechanism and Kingdee distributed file service, which is deployed across the data center cluster, which is redesigned by distributed architecture. By introducing file resource pool, file diversity between different clusters can be shielded, and transparent service can be provided for multiple applications. By introducing flexible storage and optimal storage strategy, we can support dynamic networking of multiple HDFS clusters. The cluster storage efficiency is improved by ensuring cluster resource security redundancy and nearby service. The experiment and practice prove that the file resource scheduling mechanism across HDFS cluster not only solves the problem of limited scalability of HDFS cluster, but also realizes the redundant disaster preparedness of cluster files in different places through cross-data center deployment. Cross-data center load balancing and file access services effectively improve the application experience of using KDFS storage services.
【作者單位】： 金蝶軟件中國有限公司大數(shù)據(jù)云平臺部;金蝶軟件中國有限公司國家企業(yè)互聯(lián)網(wǎng)服務(wù)支撐軟件工程技術(shù)研究中心;東南大學(xué)計算機科學(xué)與工程學(xué)院;武漢大學(xué)軟件工程國家重點實驗室;
【基金】：國家“九七三”重點基礎(chǔ)研究發(fā)展規(guī)劃項目基金(2016YFB1000800) 國家云計算示范工程(發(fā)改辦高技[2011]2448號) 國家發(fā)改委大數(shù)據(jù)示范工程(發(fā)改辦高技[2014]648號) 廣東省信息產(chǎn)業(yè)發(fā)展專項(粵經(jīng)信電軟[2015]141號) 廣東省領(lǐng)軍人才計劃([2012]342號);廣東省領(lǐng)軍人才計劃(粵人才[2012]342號)資助 深圳市技術(shù)開發(fā)項目(FWY-CX20140310010238)~~
【分類號】：TP333
，

本文編號：1930203