本文選題：Hadoop + 性能測試　； 參考：《南京大學(xué)》2015年碩士論文

【摘要】：近年來,“大數(shù)據(jù)”這個詞被越來越多地提及。傳統(tǒng)的軟件架構(gòu)在處理企業(yè)大數(shù)據(jù)時越來越力不從心。在這樣的背景下,受Google Big Table技術(shù)的啟發(fā),Hadoop應(yīng)運而生。如今,各種各樣的基于Hadoop的產(chǎn)品或者開源技術(shù)層出不窮,技術(shù)發(fā)展如火如荼。然而,Hadoop產(chǎn)品的測試卻僅限于Hadoop產(chǎn)品內(nèi)置的測試工具與第三方組織開發(fā)的benchmark,鮮有對Hadoop各個組件進行可靠性測試的測試框架,這就使得用戶在使用Hadoop產(chǎn)品時,往往由于負載大而遇到系統(tǒng)宕機等不可預(yù)知的情況,這其實是Hadoop產(chǎn)品測試人員的一種失職。在這樣的背景下,某公司設(shè)計開發(fā)了Hadoop集群性能框架,而本人作為開發(fā)者一員,完成了性能測試框架的插件配置管理與可視化測試模塊的設(shè)計與實現(xiàn)。在設(shè)計的時候,使用“插件”的概念,使得框架調(diào)度與具體測試代碼實現(xiàn)物理上的分離。使用Maven技術(shù)來管理項目構(gòu)建與依賴管理,簡化項目管理,同時還可以利用Maven生命周期在項目構(gòu)建過程中加入定制化的功能,例如類生成與資源處理；使用JAXB技術(shù)來實現(xiàn)配置類的自動生成與配置的驗證解析,簡化Model類的編寫與復(fù)雜的文件解析；項目廣泛使用spring技術(shù),來管理各種Bean的自動裝配與創(chuàng)建,通過Sprin g的Application Context管理不同插件運行時的不同環(huán)境。插件配置管理模塊作為服務(wù)提供模塊,將插件從物理Bundle轉(zhuǎn)換到邏輯概念,供調(diào)度系統(tǒng)調(diào)用,使得底層信息獲取與文件操作對于上層來說是透明的；可視化測試模塊則是為測試用戶提高測試效率,將測試人員從手動配置系統(tǒng)的繁瑣的linux操作中解放出來。利用可視化測試模塊,用戶可以新建測試,查看測試進度與監(jiān)控資源狀態(tài),并可以實時查看測試日志,測試結(jié)束時,用戶可以查看測試歷史記錄,下載測試報告與日志。利用可視化測試模塊,測試用戶的測試工作變得非常簡單。
[Abstract]:In recent years, the word "big data" has been increasingly mentioned. Traditional software architecture is more and more difficult to deal with enterprise big data. Under this background, Google Big Table technology inspired Hadoop came into being. Today, a variety of Hadoop-based products or open-source technologies emerge in endlessly, technological development is in full swing. However, the testing of Hadoop products is limited to the test tools built into Hadoop products and benchmark developed by third-party organizations. There are few test frameworks for testing the reliability of Hadoop components, which makes users use Hadoop products. Often because of the heavy load and encounter system outages and other unpredictable situations, this is in fact a Hadoop product testers dereliction of duty. Under this background, a company has designed and developed the Hadoop cluster performance framework, and as a developer, I have completed the design and implementation of plug-in configuration management and visual test module of the performance testing framework. In the design, the concept of "plug-in" is used to separate the frame scheduling from the specific test code. Using Maven technology to manage project construction and dependency management, simplify project management, and add customized functions, such as class generation and resource processing, in the process of project construction using Maven lifecycle. The JAXB technology is used to realize the automatic generation of configuration classes and the verification and parsing of configuration, which simplifies the writing of Model classes and complex file parsing. Spring technology is widely used in the project to manage the automatic assembly and creation of various Bean. Through Sprin g's Application Context, different environments of different plug-in runtime are managed. As a service providing module, plug-in configuration management module converts plug-in from physical Bundle to logical concept for scheduling system call, which makes the access and file operation of the underlying information transparent to the upper layer. The visual test module is designed to improve the test efficiency for the test users and free the testers from the tedious linux operation of the manual configuration system. With visual test module, users can build new tests, check test progress and monitor resource status, and can view test log in real time. At the end of test, users can view test history and download test report and log. Using visual test module, testing user test work becomes very simple.
【學(xué)位授予單位】：南京大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2015
【分類號】：TP311.52

【參考文獻】

相關(guān)期刊論文 前5條

1 廖黎陽;李毅;;基于JAXB和ORM的XML文檔的關(guān)系化存儲與構(gòu)建[J];兵工自動化;2009年11期

2 汪洋;徐素錦;;JAXB技術(shù)的研究與應(yīng)用[J];光盤技術(shù);2008年12期

3 何清;莊福振;;大數(shù)據(jù)挖掘與云服務(wù)模式[J];高科技與產(chǎn)業(yè)化;2013年08期

4 董曉光;喻濤;;使用Maven構(gòu)建java項目[J];電子技術(shù)與軟件工程;2014年10期

5 鄧鵬;李枚毅;何誠;;Namenode單點故障解決方案研究[J];計算機工程;2012年21期

相關(guān)會議論文 前1條

1 趙炳;胥光輝;柳旭;李慧冬;;基于Hadoop平臺的軟件系統(tǒng)的測試研究[A];第十七屆全國青年通信學(xué)術(shù)年會論文集[C];2012年

相關(guān)碩士學(xué)位論文 前1條

1 劉行亮;基于J2EE平臺的Spring框架分析研究與應(yīng)用[D];武漢科技大學(xué);2006年

，

本文編號：1949849