發(fā)布時間：2018-11-03 09:05

【摘要】：隨著全球能源互聯(lián)網(wǎng)的建設(shè)以及智能電網(wǎng)建設(shè)的快速推進,大量的物聯(lián)網(wǎng)信息采集設(shè)備終端將會接入電網(wǎng),這些終端將會產(chǎn)生海量的采集數(shù)據(jù)——智能電網(wǎng)大數(shù)據(jù)。為應(yīng)對這些海量數(shù)據(jù)分析的需求,本文研究了智能電網(wǎng)大數(shù)據(jù)的流處理和批處理引擎的構(gòu)建,并在此基礎(chǔ)上完成了智能電網(wǎng)大數(shù)據(jù)在線分析與決策系統(tǒng)的設(shè)計。本文在國內(nèi)外的研究的基礎(chǔ)上研究了智能電網(wǎng)大數(shù)據(jù)的來源和分類,對智能電網(wǎng)大數(shù)據(jù)分析的主要需求進行了分析。本文對大數(shù)據(jù)相關(guān)的分布式計算理論進行了介紹,其中主要介紹了分布式計算框架MapReduce、分布式文件系統(tǒng)GFS和HDFS、分布式應(yīng)用程序協(xié)調(diào)服務(wù)Chubby和ZooKeeper、分布式資源管理框架YARN和Mesos的原理和架構(gòu),同時也介紹了Map Reduce迭代計算模型、BSP計算模型、SSP計算模型這三種分布式數(shù)據(jù)分算法基礎(chǔ)模型;隨后研究了智能電網(wǎng)大數(shù)據(jù)流處理的任務(wù)需求,對流處理的概念進行了介紹,同時研究了智能電網(wǎng)大數(shù)據(jù)流處理系統(tǒng)的需求特征,著重研究了Strom、Spark Streaming、Samza這三種流處理引擎以及其應(yīng)用場景,根據(jù)其特點和智能電網(wǎng)大數(shù)據(jù)流處理分析的特性需求本文選擇Strom作為構(gòu)建智能大數(shù)據(jù)在線分析與決策系統(tǒng)的流處理引擎,并通過基于Storm的VFDT算法在重要電力客戶供用電安全實時分析中的應(yīng)用展示了Strom在電網(wǎng)數(shù)據(jù)實時分析上的有效性,通過機器的擴展和模擬數(shù)據(jù)流的增加進行壓力測試證明了Strom流處理引擎的在智能電網(wǎng)大數(shù)據(jù)分析場景中的可擴展性;隨后,對智能電網(wǎng)大數(shù)據(jù)批處理的任務(wù)需求進行了研究,提出了利用Spark建設(shè)智能電網(wǎng)大數(shù)據(jù)批處理分析引擎的方案,并通過基于Spark的隨機森林算法在海量用電負荷數(shù)據(jù)分析中的應(yīng)用驗證了該解決方案的有效性、可擴展性;最后,在以上研究的基礎(chǔ)上對智能電網(wǎng)大數(shù)據(jù)在線分析與決策系統(tǒng)進行了詳細的需求分析,并設(shè)計了系統(tǒng)的整體架構(gòu)和每個模塊的功能,這一設(shè)計方案可為后續(xù)的軟件開發(fā)提供直接的參考依據(jù)。
[Abstract]:With the construction of the global energy Internet and the rapid development of the smart grid, a large number of Internet of things information acquisition equipment terminals will be connected to the grid, these terminals will produce a huge amount of data acquisition-big data smart grid. In order to meet the demand of these massive data analysis, this paper studies the construction of big data's flow processing and batch processing engine of smart grid, and on this basis completes the design of online analysis and decision making system for big data in smart grid. On the basis of domestic and international research, this paper studies the source and classification of big data in smart grid, and analyzes the main requirements of the analysis of the intelligent power grid big data. This paper introduces the distributed computing theory related to big data, including the distributed computing framework MapReduce, distributed file system GFS and HDFS, distributed application coordination service Chubby and ZooKeeper,. The principle and architecture of distributed resource management framework (YARN and Mesos) are introduced, and three basic models of distributed data division algorithm, namely, Map Reduce iterative computing model, BSP computing model and SSP computing model, are also introduced. Then, the task requirements of large data flow processing in smart grid and the concept of convection processing are introduced. At the same time, the demand characteristics of large data flow processing system in smart grid are studied, and the Strom,Spark Streaming, is emphatically studied. According to the characteristics of Samza and its application scenarios, this paper chooses Strom as the flow processing engine to construct the intelligent big data online analysis and decision system according to its characteristics and the characteristics of large data flow processing and analysis of smart grid. The application of VFDT algorithm based on Storm in real-time analysis of power supply and power security of important power customers shows the effectiveness of Strom in real-time analysis of power network data. The expansibility of Strom flow processing engine in the analysis scene of smart grid big data is proved by the expansion of the machine and the increase of simulated data flow. Then, the task requirement of big data batch processing in smart grid is studied, and the scheme of building a batch processing engine based on Spark is put forward. The application of stochastic forest algorithm based on Spark in the analysis of massive power load data proves the validity and expansibility of the solution. Finally, on the basis of the above research, a detailed requirement analysis of big data online analysis and decision-making system of smart grid is carried out, and the overall structure of the system and the function of each module are designed. This design can provide a direct reference for the subsequent software development.
【學(xué)位授予單位】：華北電力大學(xué)(北京)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP311.13;F426.61

【參考文獻】

相關(guān)期刊論文 前10條

1 張發(fā)揚;李玲娟;陳煜;;VFDT算法基于Storm平臺的實現(xiàn)方案[J];計算機技術(shù)與發(fā)展;2016年09期

2 Marie-Luce PICARD;潘旭陽;;從法國公共電力企業(yè)的視角看大數(shù)據(jù)帶來的挑戰(zhàn)和機遇[J];電網(wǎng)技術(shù);2015年11期

3 鄭海雁;金農(nóng);季聰;熊政;李昆明;;電力用戶用電數(shù)據(jù)分析技術(shù)及典型場景應(yīng)用[J];電網(wǎng)技術(shù);2015年11期

4 王德文;孫志偉;;電力用戶側(cè)大數(shù)據(jù)分析與并行負荷預(yù)測[J];中國電機工程學(xué)報;2015年03期

5 張東霞;苗新;劉麗平;張焰;劉科研;;智能電網(wǎng)大數(shù)據(jù)技術(shù)發(fā)展研究[J];中國電機工程學(xué)報;2015年01期

6 劉科研;盛萬興;張東霞;賈東梨;胡麗娟;何開元;;智能配電網(wǎng)大數(shù)據(jù)應(yīng)用需求和場景分析研究[J];中國電機工程學(xué)報;2015年02期

7 宋亞奇;周國亮;朱永利;李莉;王劉旺;王德文;;云平臺下輸變電設(shè)備狀態(tài)監(jiān)測大數(shù)據(jù)存儲優(yōu)化與并行處理[J];中國電機工程學(xué)報;2015年02期

8 欒文鵬;余貽鑫;王兵;;AMI數(shù)據(jù)分析方法[J];中國電機工程學(xué)報;2015年01期

9 曲朝陽;陳帥;楊帆;朱莉;;基于云計算技術(shù)的電力大數(shù)據(jù)預(yù)處理屬性約簡方法[J];電力系統(tǒng)自動化;2014年08期

10 楊勁鋒;劉濤;陳啟冠;闕華坤;肖勇;;基于海量計量數(shù)據(jù)的電力客戶在線分群研究[J];華東電力;2013年08期

相關(guān)碩士學(xué)位論文 前6條

1 李勁松;一種基于Storm的分布式實時增量計算框架的研究與實現(xiàn)[D];電子科技大學(xué);2015年

2 馮懿;基于云計算的電力系統(tǒng)不良數(shù)據(jù)辨識算法研究[D];南京理工大學(xué);2013年

3 湛維明;云計算環(huán)境下的發(fā)電優(yōu)化調(diào)度并行算法研究[D];華北電力大學(xué);2013年

4 白紅偉;基于云計算的電力設(shè)備狀態(tài)監(jiān)測數(shù)據(jù)的存儲與查詢[D];華北電力大學(xué);2012年

5 趙黎斌;面向云存儲的分布式文件系統(tǒng)關(guān)鍵技術(shù)研究[D];西安電子科技大學(xué);2011年

6 劉芳;基于數(shù)據(jù)挖掘的電網(wǎng)數(shù)據(jù)智能分析的研究與實現(xiàn)[D];西北大學(xué);2008年

，

本文編號：2307327