【摘要】：隨著互聯(lián)網(wǎng)的高速發(fā)展,網(wǎng)絡(luò)應(yīng)用和網(wǎng)絡(luò)流量不斷增長(zhǎng),給人類社會(huì)生活、經(jīng)濟(jì)帶來(lái)便利的同時(shí),也給網(wǎng)絡(luò)管理和網(wǎng)絡(luò)安全帶來(lái)巨大的挑戰(zhàn)。通聯(lián)日志是網(wǎng)絡(luò)會(huì)話產(chǎn)生的日志,能很好地從會(huì)話級(jí)別描述網(wǎng)絡(luò)。因此如何為高速、長(zhǎng)期的通聯(lián)日志提供可靠存儲(chǔ),如何基于通聯(lián)日志準(zhǔn)確地對(duì)IP社會(huì)屬性進(jìn)行標(biāo)定,對(duì)于網(wǎng)絡(luò)安全、網(wǎng)絡(luò)管理以及網(wǎng)絡(luò)規(guī)劃都起著至關(guān)重要的作用。 現(xiàn)存的面向通聯(lián)日志的存儲(chǔ)方案沒(méi)有很好地兼顧接收、存儲(chǔ)和檢索等方面,使得它們的可移植性受到限制。多個(gè)骨干網(wǎng)生成的通聯(lián)日志每秒鐘可以達(dá)到千萬(wàn)級(jí)別,這使得傳統(tǒng)集中式存儲(chǔ)方案越來(lái)越不能滿足需求。而后的一些方案借助分布式框架提供可擴(kuò)展的存儲(chǔ)性能,但是這些方案的存儲(chǔ)引擎大多基于傳統(tǒng)關(guān)系型數(shù)據(jù)庫(kù),存儲(chǔ)性能有限。本文對(duì)通聯(lián)日志存儲(chǔ)方案進(jìn)行深入學(xué)習(xí),以實(shí)現(xiàn)支持高速存儲(chǔ)、高速查詢的分布式通聯(lián)日志存儲(chǔ)系統(tǒng)。另外,本文對(duì)傳統(tǒng)IP社會(huì)屬性標(biāo)定進(jìn)行深入研究,發(fā)現(xiàn)傳統(tǒng)的基于端口和行為特征的IP社會(huì)屬性標(biāo)定分辨率較低。本文對(duì)通聯(lián)日志進(jìn)行詳細(xì)分析,以實(shí)現(xiàn)更加準(zhǔn)確的IP社會(huì)屬性標(biāo)定。本文的主要的研究?jī)?nèi)容可以歸納為： (1)提出新型的高速通聯(lián)日志接收框架：DPIO (Driect Packet I/O)。盡管基于傳統(tǒng)的Socket API可以比較簡(jiǎn)單實(shí)現(xiàn)通聯(lián)日志接收,但是其性能不高。而新型網(wǎng)絡(luò)驅(qū)動(dòng)netmap,可以很好地解決這個(gè)問(wèn)題,但是netmap需要單獨(dú)維護(hù)網(wǎng)卡驅(qū)動(dòng),實(shí)現(xiàn)和維護(hù)都比較困難。本文在它們的基礎(chǔ)上提出一種新型的通聯(lián)日志接收框架,實(shí)驗(yàn)結(jié)果表明DPIO既能解決Socket API接收速率低的問(wèn)題,同時(shí)也能避免netmap的復(fù)雜性。 (2)設(shè)計(jì)并實(shí)現(xiàn)支持高速存儲(chǔ)和快速檢索的分布式通聯(lián)日志存儲(chǔ)系統(tǒng)：DCLStore (A Distributed Connection Log Storage System Supports High-speed storage and Fast Retrieval)。DCLStore能夠?yàn)橥?lián)日志提供高速存儲(chǔ)能力、高速檢索能力,并且通過(guò)存儲(chǔ)節(jié)點(diǎn)的動(dòng)態(tài)增加提供可擴(kuò)展的存儲(chǔ)空間。實(shí)驗(yàn)結(jié)果,本系統(tǒng)每秒鐘可以接收大約2000萬(wàn)條通聯(lián)日志,并能很好的處理多個(gè)網(wǎng)絡(luò)節(jié)點(diǎn)的日志。在查詢時(shí)可以提供比相同存儲(chǔ)容量下的單點(diǎn)存儲(chǔ)系統(tǒng)高40倍的查詢響應(yīng)速度。 (3)提出新型的IP指標(biāo)：IP明暗度。傳統(tǒng)的基于端口和行為特征的IP社會(huì)屬性標(biāo)定雖然實(shí)現(xiàn)簡(jiǎn)單,但是準(zhǔn)確率不高。本文首先對(duì)通聯(lián)日志進(jìn)行深入觀察,提出一種新型的IP指標(biāo)：IP明暗度。而后,本文對(duì)全網(wǎng)IP的指標(biāo)進(jìn)行基礎(chǔ)測(cè)量,測(cè)量結(jié)果表明本文對(duì)通聯(lián)日志處理的結(jié)果基本正確。最后,本文利用開(kāi)源工具對(duì)全網(wǎng)IP的明暗度進(jìn)行計(jì)算,并考察其對(duì)IP屬性標(biāo)定的影響。實(shí)驗(yàn)結(jié)果表明,IP明暗度對(duì)基于端口和行為特征的IP社會(huì)屬性標(biāo)定的結(jié)果都有很大的影響。
[Abstract]:With the rapid development of the Internet, network application and network traffic are increasing, which brings convenience to human life and economy, and also brings great challenges to network management and network security. The communication log is the log generated by the network session and can describe the network from the session level. Therefore, how to provide reliable storage for high-speed, long-term communication logs, and how to accurately calibrate the social attributes of IP based on communication logs play an important role in network security, network management and network planning. The existing storage schemes for communication logs do not take account of reception, storage and retrieval, so their portability is limited. Communication logs generated by multiple backbone networks can reach tens of millions of levels per second, which makes the traditional centralized storage scheme more and more unable to meet the requirements. Then some schemes provide extensible storage performance with the help of distributed framework, but most of the storage engines of these schemes are based on traditional relational databases, and the storage performance is limited. In order to realize the distributed communication log storage system which supports high speed storage and high speed query, this paper makes a deep study on the communication log storage scheme. In addition, the traditional IP social attribute calibration is deeply studied in this paper. It is found that the traditional IP social attribute calibration based on port and behavior features has lower resolution. In this paper, the connection log is analyzed in detail in order to achieve more accurate IP social attribute calibration. The main research contents of this paper can be summarized as follows: (1) A new high-speed communication log receiving framework,: DPIO (Driect Packet I / O, is proposed. Although the traditional Socket API can easily realize the communication log reception, but its performance is not high. The new network driver netmap, can solve this problem well, but the netmap needs to maintain the network card driver alone, so it is difficult to implement and maintain. In this paper, we propose a new communication log receiving framework based on them. The experimental results show that DPIO can not only solve the problem of low receiving rate of Socket API, but also avoid the complexity of netmap. (2) Design and implement the distributed connection log storage system (: DCLStore (A Distributed Connection Log Storage System Supports High-speed storage and Fast Retrieval). DCLStore), which supports high speed storage and fast retrieval, can provide high speed storage capacity and high speed retrieval ability for communication log. And provides the extensible storage space through the dynamic increase of the storage node. Experimental results show that the system can receive about 20 million communication logs per second, and can handle the logs of multiple network nodes well. The query response speed is 40 times faster than the single point storage system with the same storage capacity. (3) A new IP index, IP lightness and darkness, is proposed. The traditional IP social attribute calibration based on port and behavior is simple, but the accuracy is not high. In this paper, a new IP index, IP lightness and darkness, is proposed. Then, this paper carries on the basic measurement to the whole network IP index, the measurement result shows that the result of this paper is basically correct to the connection log processing. Finally, the open source tools are used to calculate the lightness of IP in the whole network and its influence on IP attribute calibration is investigated. The experimental results show that the IP lightness has great influence on the results of IP social attribute calibration based on port and behavioral characteristics.
【學(xué)位授予單位】：北京郵電大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2015
【分類號(hào)】：TP333

【參考文獻(xiàn)】

相關(guān)期刊論文 前2條

1 李現(xiàn)艷;趙書(shū)俊;初元萍;;基于MySQL的數(shù)據(jù)庫(kù)服務(wù)器性能測(cè)試[J];核電子學(xué)與探測(cè)技術(shù);2011年01期

2 徐非,楊廣文,鞠大鵬;基于Peer-to-Peer的分布式存儲(chǔ)系統(tǒng)的設(shè)計(jì)[J];軟件學(xué)報(bào);2004年02期

，

本文編號(hào)：2322178