發(fā)布時間：2018-05-20 00:06

  本文選題：日志分析 + 儀表盤��； 參考：《北京交通大學(xué)》2017年碩士論文

【摘要】：隨著企業(yè)信息化的不斷發(fā)展,運行的信息系統(tǒng)不斷增加,大型企業(yè)內(nèi)部部署的各種軟硬件資源不斷增加。在企業(yè)系統(tǒng)維護的過程中,數(shù)據(jù)處理的壓力會越來越大,系統(tǒng)維護的工作量和難度不斷增加。路由器,交換機,PC服務(wù)器,阿里云服務(wù)器,各種大型應(yīng)用系統(tǒng),數(shù)據(jù)庫等計算機網(wǎng)絡(luò)設(shè)備產(chǎn)生的網(wǎng)絡(luò)日志,系統(tǒng)日志,各種應(yīng)用程序日志不斷增加。在日志分析方面,傳統(tǒng)的技術(shù)在處理大量復(fù)雜日志時就會出現(xiàn)非常大的計算瓶頸。日志,作為對系統(tǒng)應(yīng)用,網(wǎng)絡(luò)設(shè)備,用戶操作的記錄工具,在保障系統(tǒng)安全性,排除系統(tǒng)故障,監(jiān)控系統(tǒng)和網(wǎng)絡(luò)運營情況等很多方面都起到舉足輕重的作用。通過日志分析可以總結(jié)查看出系統(tǒng)故障出處,用戶操作行為,設(shè)備運行狀態(tài)等很多信息。如通過查看用戶注冊登錄記錄,用戶事件記錄即可發(fā)現(xiàn)用戶是否為正常狀態(tài)。對網(wǎng)絡(luò)配置,設(shè)備參數(shù),接口變動等行為的記錄分析,就能獲取網(wǎng)絡(luò)運行狀態(tài)的信息。如果能對實時產(chǎn)生的日志做到實時全量采集,就可以進行數(shù)據(jù)實時分析或設(shè)置數(shù)據(jù)告警來對系統(tǒng)運行狀態(tài)進行監(jiān)控。通過數(shù)據(jù)儀表盤展示分析結(jié)果,是企業(yè)極易接受的一種數(shù)據(jù)展現(xiàn)形式,更能清晰的體現(xiàn)出數(shù)據(jù)的價值。本文所描述的項目是作者在北京久其智通數(shù)據(jù)科技有限公司實習(xí)過程中參與設(shè)計開發(fā)的實際項目。本人在項目中參與了系統(tǒng)需求分析,設(shè)計開發(fā)及測試部署的整個過程,主要包括系統(tǒng)前期需求調(diào)研,日志數(shù)據(jù)接入系統(tǒng)、日志檢索、可視化儀表盤和告警共四個模塊的設(shè)計,以及除了日志檢索模塊之外的開發(fā)工作。實現(xiàn)了對接入到系統(tǒng)的日志數(shù)據(jù)實時采集,實時檢索,定時監(jiān)控告警,日志分析結(jié)果可視化等功能。本論文首先根據(jù)實際業(yè)務(wù)需求對系統(tǒng)進行需求分析,確定了系統(tǒng)的功能需求和非功能需求。在此基礎(chǔ)上完成系統(tǒng)的概要設(shè)計,使用阿里云服務(wù)器作為開發(fā)的測試環(huán)境,并將系統(tǒng)分為檢索模塊,入數(shù)據(jù)到中間層模塊,中間層數(shù)據(jù)接入到ElasticSearch索引模塊,儀表盤模塊,告警模塊。完成數(shù)據(jù)庫設(shè)計后,在系統(tǒng)詳細(xì)設(shè)計與實現(xiàn)部分,對負(fù)責(zé)實現(xiàn)模塊進行功能描述和類設(shè)計,并對負(fù)責(zé)的重點功能的實現(xiàn)邏輯及實現(xiàn)過程中各方法的調(diào)用關(guān)系和實際運行效果進行了詳細(xì)的說明。系統(tǒng)上線后,針對接入系統(tǒng)的日志數(shù)據(jù)的實時快速檢索,大大提升了運維人員的工作效率,也節(jié)省了工作人員的時間成本。通過監(jiān)控告警和儀表盤的應(yīng)用,使日志數(shù)據(jù)中蘊含的豐富資源得以展現(xiàn),也大大提升了被監(jiān)控系統(tǒng)和設(shè)備的運行穩(wěn)定性,能夠?qū)ο到y(tǒng)中出現(xiàn)的問題迅速定位。
[Abstract]:With the continuous development of enterprise information, the number of information systems is increasing, and the software and hardware resources deployed in large enterprises are increasing. In the process of enterprise system maintenance, the pressure of data processing will be more and more, and the workload and difficulty of system maintenance will increase. Routers, switches, PC servers, Ali Cloud servers, a variety of large application systems, databases and other computer network equipment generated network logs, system logs, various application logs continue to increase. In the aspect of log analysis, the traditional technology will have a very large computing bottleneck when dealing with a large number of complex logs. Log, as a recording tool for system application, network equipment and user operation, plays an important role in ensuring system security, troubleshooting system, monitoring system and network operation. Through log analysis, we can summarize and view a lot of information, such as system fault source, user operation behavior, equipment running state and so on. If you view the user login record, the user event record can find out whether the user is in normal state. The information of network running state can be obtained by analyzing the behavior of network configuration, device parameters, interface change and so on. If the log generated in real time can be collected in real time, the data can be analyzed in real time or alarm can be set to monitor the running state of the system. Display the analysis result through the data dashboard, is a kind of data display form that the enterprise is easy to accept, can manifest the value of the data more clearly. The project described in this paper is a practical project in which the author participated in the design and development of the intern in Beijing Jiuqi Zhitong data Technology Co., Ltd. In the project, I participated in the whole process of system requirement analysis, design, development and test deployment, mainly including the design of four modules: system pre-requirement investigation, log data access system, log retrieval, visual dashboard and alarm. In addition to log retrieval module and the development work. The functions of real-time log data acquisition, real-time retrieval, timing monitoring and alarm monitoring, and visualization of log analysis results are realized. In this paper, the functional and non-functional requirements of the system are determined by analyzing the requirements of the system according to the actual business requirements. On this basis, the outline design of the system is completed, and the system is divided into retrieval module, data into the middle layer module, middle layer data access to ElasticSearch index module, instrument board module, using Ali cloud server as the test environment. Alarm module. After the completion of database design, in the system detailed design and implementation part, responsible for the implementation module functional description and class design, The realization logic of the key functions and the relation between the methods in the process and the actual running effect are explained in detail. After the system goes online, the real-time and fast retrieval of log data for the access system can greatly improve the efficiency of the operation and maintenance personnel, and also save the staff time cost. Through the application of monitoring alarm and dashboard, the rich resources contained in log data can be displayed, and the stability of the monitored system and equipment can be greatly improved, and the problems in the system can be quickly located.
【學(xué)位授予單位】：北京交通大學(xué)
【學(xué)位級別】：碩士
【學(xué)位授予年份】：2017
【分類號】：TP311.52

【參考文獻】

相關(guān)期刊論文 前10條

1 鄭幸源;洪親;蔡堅勇;陳順凡;柯俊敏;;基于AJAX異步傳輸技術(shù)與Echarts3技術(shù)的動態(tài)數(shù)據(jù)繪圖實現(xiàn)[J];軟件導(dǎo)刊;2017年03期

2 王仲生;;基于kafka消息隊列的文本處理技術(shù)研究[J];軟件導(dǎo)刊(教育技術(shù));2016年12期

3 徐丹;萬娜;韓貞輝;韓艷杰;王斐斐;;地震應(yīng)急數(shù)據(jù)公眾服務(wù)平臺設(shè)計[J];地震地磁觀測與研究;2016年05期

4 劉瑜君;李玉江;;論科技論文中的圖表編校[J];湖北師范學(xué)院學(xué)報(自然科學(xué)版);2016年03期

5 楊慶生;;傳輸網(wǎng)管告警短信提示系統(tǒng)研究[J];通訊世界;2016年11期

6 陳麗萍;李岑;;電子商務(wù)環(huán)境下數(shù)據(jù)安全性研究[J];電腦知識與技術(shù);2015年18期

7 王震;陳亮;;基于Kafka消息隊列的電網(wǎng)設(shè)備準(zhǔn)實時數(shù)據(jù)接入方法研究[J];山東電力技術(shù);2015年06期

8 丁晶晶;張衛(wèi)明;翟雙燦;胡晨駿;龔慶悅;;利用Ajax技術(shù)異步處理web服務(wù)中JSON格式數(shù)據(jù)的傳遞[J];信息與電腦(理論版);2013年09期

9 郭海偉;;基于注解的Spring3 MVC程序開發(fā)[J];電腦編程技巧與維護;2013年15期

10 彭霞;朱萍;任永昌;;軟件詳細(xì)設(shè)計工具對比分析研究[J];計算機技術(shù)與發(fā)展;2013年03期

相關(guān)博士學(xué)位論文 前1條

1 曹琿;可信軟件安全性測評關(guān)鍵理論及技術(shù)研究[D];武漢大學(xué);2012年

相關(guān)碩士學(xué)位論文 前4條

1 仇臣;Docker容器的性能監(jiān)控和日志服務(wù)的設(shè)計與實現(xiàn)[D];浙江大學(xué);2016年

2 趙晉萍;基于LAMP框架的通用數(shù)據(jù)測試平臺的設(shè)計與實現(xiàn)[D];北京交通大學(xué);2015年

3 劉響;廣域監(jiān)控系統(tǒng)功能測試平臺的設(shè)計與開發(fā)[D];南京理工大學(xué);2009年

4 黃浪;企業(yè)級的B/S模式應(yīng)用軟件非功能性需求分析與研究[D];廈門大學(xué);2008年

，

本文編號：1912287