本文關(guān)鍵詞：面向油田科技項(xiàng)目管理的大數(shù)據(jù)查詢優(yōu)化研究　出處：《東北石油大學(xué)》2017年碩士論文　論文類型：學(xué)位論文

  更多相關(guān)文章： 科技項(xiàng)目管理 大數(shù)據(jù) ETL設(shè)計(jì) 查詢優(yōu)化

【摘要】：2009年大慶油田科技管理平臺(tái)開始上線運(yùn)行,平臺(tái)實(shí)現(xiàn)了對(duì)科技項(xiàng)目審核的全生命周期管理。隨著時(shí)代的變遷,有關(guān)油田科技項(xiàng)目的數(shù)據(jù)量呈逐年增長(zhǎng)態(tài)勢(shì),龐大的數(shù)據(jù)量給科技項(xiàng)目管理增加了諸多龐雜的工作,為科技項(xiàng)目的高效管理和避免重復(fù)立項(xiàng)等情況的發(fā)生,如何能夠快速和正確的判斷別相似的海量數(shù)據(jù)信息是要解決的首要問題。因此,本課題擬引入大數(shù)據(jù)的相關(guān)概念,通過搭建大數(shù)據(jù)運(yùn)行環(huán)境,應(yīng)用大數(shù)據(jù)的相關(guān)技術(shù)對(duì)油田科技項(xiàng)目的查詢進(jìn)行優(yōu)化研究,以此來提高油田項(xiàng)目管理的效率。本文基于課題研究的背景及實(shí)際需求,首先,構(gòu)建了Hadoop大數(shù)據(jù)運(yùn)行環(huán)境,針對(duì)油田科技項(xiàng)目的數(shù)據(jù)類型和數(shù)據(jù)源,展開了存儲(chǔ)結(jié)構(gòu)的研究和搭建,提供了對(duì)異地?cái)?shù)據(jù)源的數(shù)據(jù)進(jìn)行加載存儲(chǔ)的策略,以及針對(duì)多種類型數(shù)據(jù)中的結(jié)構(gòu)化數(shù)據(jù)展開研究,設(shè)計(jì)支持向存儲(chǔ)后的數(shù)據(jù)進(jìn)行ETL處理的NoSQL數(shù)據(jù)庫(kù)。其次,在此基礎(chǔ)上建立基于Hadoop平臺(tái)的ETL體系結(jié)構(gòu),實(shí)現(xiàn)更為合理的數(shù)據(jù)ETL工作流程,為下一步的查詢優(yōu)化提供保證。最后,針對(duì)油田科技項(xiàng)目管理的數(shù)據(jù)查詢,結(jié)合當(dāng)前大數(shù)據(jù)查詢方法的弊端,分析并研究其底層運(yùn)行機(jī)制和流程,展開優(yōu)化設(shè)計(jì),從而提高整個(gè)大數(shù)據(jù)查詢系統(tǒng)的效率,較好的解決了油田科技項(xiàng)目在面向海量數(shù)據(jù)時(shí)的管理效率問題。本課題針對(duì)油田科技項(xiàng)目管理平臺(tái)的實(shí)際發(fā)展及需要,將基于大數(shù)據(jù)環(huán)境下的存儲(chǔ)結(jié)構(gòu),查詢方法融合于油田科技項(xiàng)目管理上,并開發(fā)了相應(yīng)的系統(tǒng)進(jìn)行實(shí)驗(yàn),驗(yàn)證其可行性,為課題的實(shí)現(xiàn)提供了理論支撐和實(shí)例驗(yàn)證,在一定程度上對(duì)數(shù)據(jù)量日益增長(zhǎng)的油田科技項(xiàng)目管理具有一定的理論意義與參考價(jià)值。
[Abstract]:In 2009, Daqing oilfield science and technology management platform began to run online, the platform realized the full life cycle management of science and technology project audit. The volume of scientific and technological projects in oilfields is increasing year by year, and the huge amount of data adds a lot of complicated work to the management of scientific and technological projects, which is the occurrence of the efficient management of scientific and technological projects and the avoidance of duplicate projects. How to quickly and correctly judge the similar mass of data information is the first problem to be solved. Therefore, this paper intends to introduce the relevant concepts of big data, through the establishment of big data operating environment. In order to improve the efficiency of oil field project management, big data's related technology is used to optimize the query of oilfield science and technology project. Firstly, based on the background of the research and the actual needs. The Hadoop big data environment is constructed, and the storage structure is studied and built according to the data types and data sources of oilfield science and technology projects. The strategy of loading and storing data from remote data sources and the research of structured data in various types of data are provided. Design the NoSQL database which supports the ETL processing to the stored data. Secondly, build the ETL architecture based on the Hadoop platform. To achieve a more reasonable data ETL workflow, for the next step of query optimization to provide assurance. Finally, for oilfield science and technology project management data query, combined with the current shortcomings of big data query method. Analysis and study of its underlying operation mechanism and process, the development of optimization design, so as to improve the efficiency of the whole big data query system. Better solve the problem of management efficiency of oilfield science and technology projects in the face of massive data. This subject will be based on the storage structure of big data environment in view of the actual development and needs of oilfield science and technology project management platform. The query method is integrated into the oil field science and technology project management, and the corresponding system is developed to carry on the experiment, to verify its feasibility, provides the theory support and the example verification for the realization of the subject. To a certain extent, it has certain theoretical significance and reference value for the management of oil field science and technology project.
【學(xué)位授予單位】：東北石油大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2017
【分類號(hào)】：TE4;TP311.13

【相似文獻(xiàn)】

中國(guó)期刊全文數(shù)據(jù)庫(kù) 前10條

1 岳耀書，蔣誕一，向新柱;確定科技項(xiàng)目的基本模式研究[J];武漢紡織工學(xué)院學(xué)報(bào);1994年S1期

2 趙明;在科技項(xiàng)目中的應(yīng)用加強(qiáng)項(xiàng)目管理[J];石油規(guī)劃設(shè)計(jì);2002年02期

3 李家慶;孫宴增;盧明;關(guān)新;吳兵;;科技項(xiàng)目管理更要科學(xué)化[J];中國(guó)石油企業(yè);2006年11期

4 胡克松;;石油企業(yè)科技項(xiàng)目管理存在的問題及對(duì)策研究[J];經(jīng)濟(jì)研究導(dǎo)刊;2012年12期

5 帥訓(xùn)波;李家慶;葛蘭;霍思，

本文編號(hào)：1396196