【摘要】：隨著微博、社交網絡、移動APP、基于位置的服務等技術的興起,全球數據出現了爆炸式的增長。大數據為數據的挖掘和分析提供了豐富的來源,從中挖掘出來的結果將變得更有意義甚至充滿驚喜。然而面對大數據傳統(tǒng)的處理方法和處理能力都漸漸變得力不從心。如何高效的進行大數據的處理已經成為各行各業(yè)爭相研究的熱點問題。本文從影視大數據智能分析系統(tǒng)建設過程中遇到的大數據處理的問題出發(fā),分析研究了現有的數據處理方案,結合了云計算和大數據處理的相關處理技術,提出了影視大數據處理策略及方法。主要研究主要內容包括以下幾個方面,首先對云計算的概念和相關技術進行研究。選定了 Hadoop開源云平臺作為項目研究的基礎平臺,深入研究了 MapReduce分布式編程模型以及HDFS分布式文件存儲系統(tǒng)。其次深入分析了影視數據各部分的業(yè)務需求,為應對不同的數據處理需求,充分利用關系型數據庫和Hadoop分布式平臺的優(yōu)點,提出了兩種數據處理方案。一種基于關系型數據庫的單節(jié)點數據處理,一種基于Hadoop的分布式數據處理。最后根據影視大數據智能分析系統(tǒng)不同的數據特點和不同的數據處理需求,選擇合適的數據處理方案進行數據處理流程的設計和實現,并成功的應用于影視大數據的處理中,為影視大數據分析系統(tǒng)提供了可靠的數據支持。
[Abstract]:With the rise of technologies such as Weibo, social networks and mobile APP, location-based services, there has been explosive growth in global data. Big data provides a rich source for data mining and analysis, from which the results will become more meaningful and even full of surprises. However, in the face of big data's traditional treatment methods and processing capacity have gradually become unable to do so. How to deal with big data efficiently has become a hot issue in various industries. Starting from the problems encountered by big data in the construction of big data intelligent analysis system for film and television, this paper analyzes and studies the existing data processing schemes, and combines the related processing technologies of cloud computing and big data processing. The treatment strategy and method of big data in film and television are put forward. The main research contents include the following aspects, first of all, the concept of cloud computing and related technologies are studied. The Hadoop open source cloud platform is selected as the basic platform of project research, and the MapReduce distributed programming model and HDFS distributed file storage system are deeply studied. Secondly, the business requirements of each part of film and television data are deeply analyzed. In order to meet the different data processing requirements and make full use of the advantages of relational database and Hadoop distributed platform, two data processing schemes are proposed. A single node data processing based on relational database and a distributed data processing based on Hadoop. Finally, according to the different data characteristics and different data processing requirements of the film and television big data intelligent analysis system, the appropriate data processing scheme is selected to design and implement the data processing flow, and it is successfully applied to the processing of film and television big data. It provides reliable data support for the film and television big data analysis system.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2016
【分類號】：TP311.13

【參考文獻】

相關期刊論文 前10條

1 劉圓;王峰;楊明川;;面向大數據的分布式存儲技術研究[J];電信技術;2015年06期

2 程學旗;靳小龍;王元卓;郭嘉豐;張鐵贏;李國杰;;大數據系統(tǒng)和分析技術綜述[J];軟件學報;2014年09期

3 唐世慶;李云龍;田鳳明;胡海榮;;基于Hadoop的云計算與存儲平臺研究與實現[J];四川兵工學報;2014年08期

4 馮登國;張敏;李昊;;大數據安全與隱私保護[J];計算機學報;2014年01期

5 陳忻;;暢游數據海洋[J];中國建設信息;2013年06期

6 張松林;;云計算的核心技術與應用實例[J];電子世界;2013年05期

7 張耀祥;;云計算和虛擬化技術[J];計算機安全;2011年05期

8 詹洪文;;云計算核心技術及其產業(yè)化淺析[J];硅谷;2011年06期

9 李成華;張新訪;金海;向文;;MapReduce:新型的分布式并行計算編程模型[J];計算機工程與科學;2011年03期

10 胡云;;對云計算技術及應用的研究[J];電腦開發(fā)與應用;2011年03期

相關碩士學位論文 前1條

1 叢中昌;基于云計算平臺的電信經營分析系統(tǒng)中海量數據處理研究[D];南京郵電大學;2011年

，

本文編號：2485057