本文選題：流數(shù)據(jù) + 數(shù)據(jù)通信�。� 參考：《北方工業(yè)大學(xué)》2016年碩士論文

【摘要】：隨著物聯(lián)網(wǎng)技術(shù)的普及,催生了很多新的應(yīng)用,同時(shí)給傳統(tǒng)行業(yè)帶來(lái)了變化,例如智能家居、智能交通、智慧城市等。不同行業(yè)的行為活動(dòng)已經(jīng)被轉(zhuǎn)變?yōu)閯?dòng)態(tài)流數(shù)據(jù),這些數(shù)據(jù)具有實(shí)時(shí)性、并發(fā)性、異構(gòu)性等新特點(diǎn),傳統(tǒng)的處理方法和處理模型已經(jīng)不能夠滿(mǎn)足于動(dòng)態(tài)流數(shù)據(jù)的接入需求。流數(shù)據(jù)接入的主要難點(diǎn)在于數(shù)據(jù)種類(lèi)多樣和數(shù)據(jù)來(lái)源廣泛：例如有GPS數(shù)據(jù)、交通車(chē)輛數(shù)據(jù)、傳感器采集的數(shù)據(jù)以及其他感知數(shù)據(jù)等類(lèi)型；數(shù)據(jù)來(lái)源包括交通、農(nóng)業(yè)、醫(yī)療、電商等各個(gè)領(lǐng)域來(lái)自于大量的前端傳感器和不同行為轉(zhuǎn)化的數(shù)據(jù)。在上述數(shù)據(jù)接入的技術(shù)需求下,產(chǎn)生了兩方面的問(wèn)題：動(dòng)態(tài)流數(shù)據(jù)怎么被接入到云中心,不同類(lèi)型的動(dòng)態(tài)流數(shù)據(jù)如何傳輸和大量前端的異構(gòu)數(shù)據(jù)怎么接入；如何滿(mǎn)足流數(shù)據(jù)通信服務(wù)器對(duì)于數(shù)據(jù)的高效解析和分發(fā),為了應(yīng)對(duì)大量數(shù)據(jù)的接入和支撐其它業(yè)務(wù)的處理,需要高效的解析數(shù)據(jù)和分發(fā)數(shù)據(jù)。本文針對(duì)現(xiàn)在缺少有效的流數(shù)據(jù)匯集系統(tǒng),在掌握了網(wǎng)絡(luò)通信協(xié)議、異構(gòu)流數(shù)據(jù)匯集技術(shù)以及異步10技術(shù)后,設(shè)計(jì)并實(shí)現(xiàn)了一個(gè)支持多源異構(gòu)流數(shù)據(jù)匯集的通信服務(wù)器,我們稱(chēng)之為流數(shù)據(jù)通信服務(wù)器。在云環(huán)境下可以支持海量數(shù)據(jù)的接入、解析和分發(fā)。并且為用戶(hù)提供web端,可以監(jiān)測(cè)數(shù)據(jù)變化,同時(shí)對(duì)數(shù)據(jù)進(jìn)行持久化,以備其它應(yīng)用可以使用流數(shù)據(jù)。論文的主要內(nèi)容包括：(I)設(shè)計(jì)了一種支持異構(gòu)流數(shù)據(jù)匯集的數(shù)據(jù)通信協(xié)議,包含了發(fā)送的數(shù)據(jù)幀格式、響應(yīng)的數(shù)據(jù)幀格式以及增強(qiáng)可靠性的一些設(shè)計(jì)。把不同來(lái)源、不同類(lèi)型的數(shù)據(jù)按照數(shù)據(jù)通信協(xié)議的格式打包再接入系統(tǒng),不僅實(shí)現(xiàn)了數(shù)據(jù)通信而且能夠支持異構(gòu)流數(shù)據(jù)的接入。(2)采用異步10方式實(shí)現(xiàn)了支持高并發(fā)的流數(shù)據(jù)通信服務(wù)器,本文主要使用了Libevent和多線(xiàn)程連接池等技術(shù)來(lái)實(shí)現(xiàn)高并發(fā)機(jī)制。在接入大量數(shù)據(jù)后能夠高并發(fā)的對(duì)數(shù)據(jù)進(jìn)行解析和分發(fā)來(lái)保證系統(tǒng)的性能,同時(shí)能夠?qū)⑻幚砗笙D(zhuǎn)發(fā)給消息服務(wù)器實(shí)現(xiàn)消息中轉(zhuǎn)。(3)開(kāi)發(fā)了一套物聯(lián)網(wǎng)感知數(shù)據(jù)托管服務(wù)應(yīng)用原型系統(tǒng),在本文已實(shí)現(xiàn)的流數(shù)據(jù)通信服務(wù)器基礎(chǔ)上設(shè)計(jì)的。能夠?qū)崿F(xiàn)用戶(hù)對(duì)設(shè)備和數(shù)據(jù)的管理,為云環(huán)境下的不同用戶(hù)的不同類(lèi)型的感知數(shù)據(jù)匯集提供了新的服務(wù)支撐模式。此外,經(jīng)實(shí)驗(yàn)測(cè)試,系統(tǒng)在普通服務(wù)器配置下可達(dá)到3000并發(fā)長(zhǎng)連接(即3000個(gè)終端設(shè)備下每秒3000條數(shù)據(jù)的接收能力)
[Abstract]:With the popularization of the Internet of things technology, many new applications have been spawned, at the same time brought changes to the traditional industries, such as smart home, intelligent transportation, intelligent city and so on. The behavioral activities of different industries have been transformed into dynamic flow data. These data have some new characteristics such as real-time, concurrency, heterogeneity and so on. The traditional processing methods and processing models can no longer meet the access requirements of dynamic flow data. The main difficulties in accessing stream data are the variety of data and the wide range of data sources, such as GPS data, traffic vehicle data, sensor data and other perceptual data, etc. The data sources include transportation, agriculture, medicine, etc. E-quotient and other fields come from a large number of front-end sensors and different behavior conversion data. Under the technical requirements of the above data access, there are two problems: how to connect the dynamic stream data to the cloud center, how to transmit the different types of dynamic flow data and how to access a large number of heterogeneous data in the front end; How to satisfy the efficient data parsing and distribution of streaming data communication server, in order to deal with the access of a large number of data and support the processing of other services, it is necessary to efficiently parse and distribute data. Aiming at the lack of effective streaming data collection system, after mastering the network communication protocol, heterogeneous stream data collection technology and asynchronous 10 technology, this paper designs and implements a communication server that supports multi-source heterogeneous stream data collection. We call it streaming data communication server. In the cloud environment can support massive data access, parsing and distribution. It can monitor the data change and persist the data in case other applications can use stream data. The main contents of this paper are as follows: (1) A data communication protocol supporting heterogeneous stream data collection is designed, which includes the data frame format sent, the response data frame format and some designs to enhance reliability. Packaging different sources and types of data into the system in the format of the data communication protocol, This paper not only realizes data communication but also supports the access of heterogeneous stream data. It uses asynchronous 10 mode to realize the high concurrency stream data communication server. This paper mainly uses Libevent and multi-thread connection pool to realize the high concurrency mechanism. After accessing a large amount of data, the data can be parsed and distributed in high concurrency to ensure the performance of the system. At the same time, we can forward the processed message packet to the message server to realize the message transfer. We have developed a prototype system of Internet of things aware data hosting service, which is designed on the basis of the stream data communication server which has been implemented in this paper. It can realize the user's management of the device and data, and provide a new service supporting mode for the different types of perceptual data collection of different users in the cloud environment. In addition, experimental tests show that the system can achieve 3000 concurrent long connections (i.e., the receiving capacity of 3000 data per second under 3,000 terminal devices) under the common server configuration.
【學(xué)位授予單位】：北方工業(yè)大學(xué)
【學(xué)位級(jí)別】：碩士
【學(xué)位授予年份】：2016
【分類(lèi)號(hào)】：TP311.13

【相似文獻(xiàn)】

相關(guān)期刊論文 前10條

1 唐云建;石為人;易軍;王燕霞;;面向WSN數(shù)據(jù)匯集應(yīng)用的動(dòng)態(tài)負(fù)載均衡算法[J];計(jì)算機(jī)工程與應(yīng)用;2011年06期

2 唐云建;石為人;易軍;王燕霞;;基于蟻群優(yōu)化的WSN負(fù)載均衡數(shù)據(jù)匯集算法[J];高技術(shù)通訊;2010年08期

3 蘇海濱;王彥超;于步亞;;省網(wǎng)小水電管理技術(shù)支持系統(tǒng)數(shù)據(jù)匯集平臺(tái)子系統(tǒng)的設(shè)計(jì)[J];自動(dòng)化應(yīng)用;2013年02期

4 朱藝華;袁婧;李燕君;彭靜;;分裂無(wú)線(xiàn)傳感器網(wǎng)絡(luò)基于消息擺渡的數(shù)據(jù)匯集策略[J];計(jì)算機(jī)研究與發(fā)展;2010年S2期

5 孫欽;張宏軍;劉耀勛;張睿;;軍事訓(xùn)練演習(xí)數(shù)據(jù)匯集與融合系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[J];指揮控制與仿真;2012年03期

6 譚明超;馮徑;舒曉村;王娟;;基于JMS的氣象數(shù)據(jù)匯集系統(tǒng)的研究與實(shí)現(xiàn)[J];計(jì)算機(jī)與信息技術(shù);2006年03期

7 劉玲;柴喬林;耿曉義;;考慮負(fù)載均衡的無(wú)線(xiàn)傳感器網(wǎng)絡(luò)數(shù)據(jù)匯集算法[J];計(jì)算機(jī)工程與應(yīng)用;2007年33期

8 石為人;唐云建;王燕霞;;基于擁塞控制的無(wú)線(xiàn)傳感器網(wǎng)絡(luò)數(shù)據(jù)匯集樹(shù)生成算法[J];自動(dòng)化學(xué)報(bào);2010年06期

9 韋啟森;貴在探索[J];上海微型計(jì)算機(jī);2000年Z2期

10 陳永燥;程紅舉;;多Sink無(wú)線(xiàn)傳感器網(wǎng)絡(luò)中最小時(shí)延數(shù)據(jù)匯集調(diào)度算法設(shè)計(jì)[J];小型微型計(jì)算機(jī)系統(tǒng);2014年04期

相關(guān)會(huì)議論文 前1條

1 黃劍;王海員;石為人;唐云建;許磊;;基于蟻群優(yōu)化的能量均衡數(shù)據(jù)匯集算法[A];2010中國(guó)儀器儀表與測(cè)控技術(shù)大會(huì)論文集[C];2010年

相關(guān)重要報(bào)紙文章 前1條

1 艾德里安·愛(ài)莎 李沐 譯;黃金數(shù)據(jù)匯集:關(guān)于金價(jià)暴跌的歷史學(xué)觀(guān)點(diǎn)[N];中國(guó)黃金報(bào);2013年

相關(guān)博士學(xué)位論文 前2條

1 唐云建;無(wú)線(xiàn)傳感器網(wǎng)絡(luò)負(fù)載均衡數(shù)據(jù)匯集算法研究[D];重慶大學(xué);2010年

2 李云鶴;基于壓縮網(wǎng)絡(luò)編碼的高能效WSN數(shù)據(jù)匯集方法研究[D];哈爾濱工業(yè)大學(xué);2014年

相關(guān)碩士學(xué)位論文 前5條

1 王曉暉;支持多源異構(gòu)流數(shù)據(jù)匯集的通信服務(wù)器設(shè)計(jì)與實(shí)現(xiàn)[D];北方工業(yè)大學(xué);2016年

2 曹張治;空間環(huán)境科學(xué)數(shù)據(jù)匯集及關(guān)鍵技術(shù)研究[D];中國(guó)地質(zhì)大學(xué)（北京）;2010年

3 王曉升;基于閃存陣列的數(shù)據(jù)匯集系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];中北大學(xué);2014年

4 雷姝潔;基于Tuxedo的水利數(shù)據(jù)匯集探析[D];南昌大學(xué);2009年

5 程峰;子午工程數(shù)據(jù)匯集子系統(tǒng)的設(shè)計(jì)與實(shí)現(xiàn)[D];中國(guó)地質(zhì)大學(xué)（北京）;2009年

，

本文編號(hào)：1887993