云環(huán)境下基于多目標優(yōu)化算法的微博意見領袖挖掘
發(fā)布時間：2019-04-10 08:31
【摘要】：近幾年,隨著互聯(lián)網(wǎng)的普及,人們的交流方式也發(fā)生了翻天覆地的變化。微博作為一種新興的網(wǎng)絡信息傳播媒體受到廣大網(wǎng)民的追捧,它的時效性和裂變性可以使信息在很短的時間內(nèi)得到廣泛的傳播,因此對信息的有效控制就變得十分重要。因為,一旦有人蓄意傳播虛假信息或者危害社會安全的言論,后果將十分嚴重。那么,對在網(wǎng)絡信息傳播中占主導地位的意見領袖的挖掘和監(jiān)控就顯得意義重大。 目前對于意見領袖的挖掘方法主要有統(tǒng)計學方法、聚類分析法、基于SNA社會網(wǎng)絡分析法等,這些方法都有各自的特點,但是面對331億微博網(wǎng)民所產(chǎn)生的海量微博數(shù)據(jù),它們并沒有表現(xiàn)出很好的處理能力。 本文以微博用戶屬性為立足點,將其多個屬性特征與多目標優(yōu)化問題結合起來,提出把Skyline查詢引入到微博意見領袖的挖掘中,Skyline查詢是解決多目標優(yōu)化問題的一類方法。面對海量的微博數(shù)據(jù),本文引入Hadoop關鍵技術MapReduce框架,將Skyline計算中的BNL塊嵌套循環(huán)算法和SFS排序過濾算法在該編程框架下實現(xiàn),使其在處理海量數(shù)據(jù)時有更好的性能。接著對意見領袖評估模型進行建立,提出從用戶影響力和用戶參與度兩個指標評價微博意見領袖,并利用AHP層次分析法確定各屬性權重,最后給出意見領袖計算公式。在挖掘實驗階段,搭建Hadoop集群環(huán)境,設計微博爬蟲獲取微博數(shù)據(jù),將數(shù)據(jù)用并行化后的SFS算法進行處理,再將處理結果運用于意見領袖模型中進行計算。最后將本文挖掘的意見領袖與新浪微博官方人氣用戶進行對比,結果顯示本文的方法挖掘出的意見領袖分布領域相對廣泛,在一定程度上避免了新浪微博官方用戶排名中娛樂人物一家獨大的現(xiàn)象。因此,本文的思路可以作為一種解決方法,處理海量、高維數(shù)據(jù),為微博意見領袖的挖掘提供了一種可能的選擇。
[Abstract]:In recent years, with the popularity of the Internet, people's way of communication has also undergone earth-shaking changes. As a new kind of network information communication media, Weibo has been pursued by many netizens, its timeliness and fragmentation can make information widely spread in a very short time, so the effective control of information becomes very important. As soon as someone deliberately spreads false information or endangers social security, the consequences will be very serious. So, it is of great significance to excavate and monitor the opinion leaders who play a dominant role in the network information dissemination. At present, there are statistical method, cluster analysis method and social network analysis method based on SNA for opinion leader mining. These methods have their own characteristics. However, in the face of the massive Weibo data generated by 33.1 billion Weibo netizens, They don't show good handling power. In this paper, taking Weibo user attribute as the foothold, combining its multi-attribute features with multi-objective optimization problem, this paper proposes to introduce Skyline query into the mining of Weibo opinion leader. Skyline query is a kind of method to solve multi-objective optimization problem. Facing the huge amount of Weibo data, this paper introduces MapReduce framework, the key technology of Skyline, and implements the BNL block nested loop algorithm and SFS sorting filtering algorithm in Skyline computing under this programming framework, so that it has better performance when dealing with massive data. Then the evaluation model of opinion leader is established, and two indexes of user influence and user participation are put forward to evaluate the opinion leader of Weibo. The weight of each attribute is determined by AHP, and the formula of opinion leader is given at last. In the mining experiment stage, the Hadoop cluster environment is built, the Weibo crawler is designed to obtain Weibo data, the data is processed by parallel SFS algorithm, and the processing results are applied to the opinion leader model for calculation. Finally, the author compares the opinion leaders excavated in this paper with the official popular users of Sina Weibo, and the results show that the opinion leaders are distributed in a relatively wide range of areas by the method of this paper. To some extent avoided Sina Weibo official user ranking entertainment figures in the big phenomenon. Therefore, the idea of this paper can be used as a solution to deal with massive, high-dimensional data, which provides a possible choice for the mining of Weibo opinion leaders.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2014
【分類號】：TP393.092
本文編號：2455648
[Abstract]:In recent years, with the popularity of the Internet, people's way of communication has also undergone earth-shaking changes. As a new kind of network information communication media, Weibo has been pursued by many netizens, its timeliness and fragmentation can make information widely spread in a very short time, so the effective control of information becomes very important. As soon as someone deliberately spreads false information or endangers social security, the consequences will be very serious. So, it is of great significance to excavate and monitor the opinion leaders who play a dominant role in the network information dissemination. At present, there are statistical method, cluster analysis method and social network analysis method based on SNA for opinion leader mining. These methods have their own characteristics. However, in the face of the massive Weibo data generated by 33.1 billion Weibo netizens, They don't show good handling power. In this paper, taking Weibo user attribute as the foothold, combining its multi-attribute features with multi-objective optimization problem, this paper proposes to introduce Skyline query into the mining of Weibo opinion leader. Skyline query is a kind of method to solve multi-objective optimization problem. Facing the huge amount of Weibo data, this paper introduces MapReduce framework, the key technology of Skyline, and implements the BNL block nested loop algorithm and SFS sorting filtering algorithm in Skyline computing under this programming framework, so that it has better performance when dealing with massive data. Then the evaluation model of opinion leader is established, and two indexes of user influence and user participation are put forward to evaluate the opinion leader of Weibo. The weight of each attribute is determined by AHP, and the formula of opinion leader is given at last. In the mining experiment stage, the Hadoop cluster environment is built, the Weibo crawler is designed to obtain Weibo data, the data is processed by parallel SFS algorithm, and the processing results are applied to the opinion leader model for calculation. Finally, the author compares the opinion leaders excavated in this paper with the official popular users of Sina Weibo, and the results show that the opinion leaders are distributed in a relatively wide range of areas by the method of this paper. To some extent avoided Sina Weibo official user ranking entertainment figures in the big phenomenon. Therefore, the idea of this paper can be used as a solution to deal with massive, high-dimensional data, which provides a possible choice for the mining of Weibo opinion leaders.
【學位授予單位】：北京郵電大學
【學位級別】：碩士
【學位授予年份】：2014
【分類號】：TP393.092
【參考文獻】
相關期刊論文 前10條
1 李名亮;;現(xiàn)實與隱憂:微博意見領袖的話語權力[J];今傳媒;2012年05期
2 桑亮;許正林;;微博意見領袖的形成機制及其影響[J];當代傳播;2011年03期
3 肖宇;許煒;夏霖;;一種基于情感傾向分析的網(wǎng)絡團體意見領袖識別算法[J];計算機科學;2012年02期
4 王玨;曾劍平;周葆華;吳承榮;;基于聚類分析的網(wǎng)絡論壇意見領袖發(fā)現(xiàn)方法[J];計算機工程;2011年05期
5 王甲生;吳曉平;廖巍;陳永強;;改進的加權復雜網(wǎng)絡節(jié)點重要度評估方法[J];計算機工程;2012年10期
6 肖宇;許煒;夏霖;;網(wǎng)絡社區(qū)中的意見領袖特征分析[J];計算機工程與科學;2011年01期
7 馬寧;劉怡君;;基于超網(wǎng)絡的輿論領袖識別應用研究[J];中國科學院院刊;2012年05期
8 張萌;余芬芬;;微博意見領袖對網(wǎng)絡輿論的影響[J];新聞世界;2012年07期
9 肖宇;許煒;張晨;何丹丹;;社交網(wǎng)絡中用戶區(qū)域影響力評估算法研究[J];微電子學與計算機;2012年07期
10 王平;謝耘耕;;突發(fā)公共事件中微博意見領袖的實證研究——以“溫州動車事故”為例[J];現(xiàn)代傳播(中國傳媒大學學報);2012年03期
，本文編號：2455648
本文鏈接：http://www.sikaile.net/guanlilunwen/ydhl/2455648.html
最近更新
教材專著
